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FOREWORD 


THE TEAGLE FOUNDATION HAS AN active program of communica- 
tion, reflecting its mission to share the knowledge that arises from its work 
and the experiences of its grantees. The Foundation’s website is an important 
part of the effort to disseminate ideas and practices and includes a wide vari- 
ety of reports, opinion pieces, and essays by Teagle staff members, grantees, 
and leaders in higher education. For the first time, a full-length book is appear- 
ing as an original publication on the Teagle website. This collection of essays, 
Literary Study, Measurement, a?id the Sublime: Disciplinary Assessment , edited by 
Donna Heiland and Laura J. Rosenthal, represents an important new venture 
in the Foundation’s communication program (it will appear in print as well, 
so that readers can engage with it in the format they prefer). The book is 
the product of many authors, including the editors, both of whom have 
written essays for it. But it is the creativity and the persistence of the edi- 
tors that explains the appearance of this new publication. The editors have 
reviewed the essays rigorously, to ensure that they meet the highest aca- 
demic standards. 

The essays represent an enticing and interesting series of ideas and experi- 
ences about the work of assessment in literature and related fields that often 
resist the language and the methods of standard forms of assessment — often, 
one might add, for very good reasons. Yet it is just the play of ideas and inter- 
pretations that one finds in literary studies that highlights the collection. The 
opinions of various authors about the uses and the abuses of assessment cover 
a wide spectrum of views that reflect the perspectives that the Foundation finds 
in its own work with its grantees. This is to be expected and even welcomed, 
for little learning occurs if answers are easy or consensus is artificial. In pub- 
lishing this work, Donna Heiland and Laura Rosenthal have contributed signif- 
icantly to an understanding of the range and the possibilities of assessment, 
and to the work of the Foundation. 

Richard L. Morrill 

President, The Teagle Foundation 
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INTRODUCTION 

DONNA HEILAND AND LAURA J. ROSENTHAL 


ASSESSMENT IS CURRENTLY RESHAPING the academic landscape. In 
a recent survey of over fifty English department chairs and graduate directors 
conducted in anticipation of the 2010 meeting of the Association of Department 
Chairs (ADE) East, 86% reported that their department was engaged in some 
form of learning outcomes assessment (“ADE Survey”). While this high per- 
centage may not necessarily obtain throughout all English and foreign lan- 
guage departments, only two percent of those who responded reported that 
they were neither engaged in assessment projects nor had plans to set one in 
motion. Of course, this simple statistic represents a wide range of practices. 
Inevitably, some departments are spending most of their energy trying to fig- 
ure out ways to generate just enough information to satisfy the demands of 
accountability; at the other end of the spectrum, some departments are seeing 
ways to use assessment to rethink student learning in their programs. Most fall 
somewhere in between. Faculty responses at most institutions inevitably vary 
as well: some instructors have embraced assessment, some are cautiously 
exploring its possibilities, and others have vehemently protested its institution- 
alization. Yet, it has begun to reach a saturation point in academic culture. 
Even Facebook now offers as one of its “Shite Gifts for Academics” an 
“Overly-Enthusiastic Assessment Guru,” presumably of the kind that no one 
wants to see at the next department meeting. Love or hate it, learning out- 
comes assessment is becoming an important part of academic life in English 
and foreign language departments. 

While assessment has become an occupational reality (and, some might 
say, an occupational hazard), in many professional venues for literary and cul- 
tural studies there has been relatively little discussion about this process and its 
implications. While professional issues are regularly and vigorously debated at 
conferences and in journals, discussions of assessment tend to be relegated to 
more administrative venues and treated as practical matters. 1 These discus- 
sions take place in meetings of campus committees, in departments, and in 
informal settings (by this point most instructors have probably opined about 
assessment around the proverbial water cooler), but often do not move much 
beyond these contexts. While most departments, then, are conducting assess- 
ment projects, and while many faculty members currently participate in those 
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projects, and while many instructors have strong opinions about assessment, 
few of the questions raised by assessment have attracted the kind of sustained 
thought that we give to other aspects of professional life. This collection of 
essays attempts to end this silence, asking first of all: if assessment has become 
so pervasive in professional academic life, then how do we account for the lim- 
its of the discussion that has so far taken place in our own disciplinary context? 

This cannot be simply because outcomes assessment is not a literary topic, 
like Shakespeare or the Harlem Renaissance, as many topics of broader pro- 
fessional interest have attracted attention in recent years. We see publications 
and conference panels on academic freedom, the exploitation of adjunct 
instructors, the tenure system, the politics of the classroom, university budgets, 
graduate student mentoring, and faculty self-governance. As conditions in 
higher education change, it often feels like these discussions have become what 
theory was in the 1 980s: the problems in which everyone has a stake and that 
in certain ways generate the most interesting fundamental discussions. Nor 
does any lack of interest in teaching or classroom issues among literary schol- 
ars explain the under-theorizing of assessment. Our primary professional 
organization, the Modern Language Association (MLA), publishes a series of 
books devoted to teaching strategies (“Approaches to Teaching”), and promi- 
nent scholars have published monographs on the teaching of literature (Graff, 
Showalter, Tompkins). Most professional organizations devoted to the academ- 
ic sub-fields in literary study dedicate conference panels to teaching practices 
and some offer prizes for course designs. Every job applicant generates a 
teaching philosophy and must sometimes even show pedagogy in action by 
teaching a sample class on the campus visit. Further, few topics in literary 
studies have attracted greater attention in the general public than what kinds 
of texts get taught. Newspapers and novelists have dissected conflicts over the 
literary canon; careers have been made and lost in their wake. 

No one can reasonably argue, then, that we overlook teaching as an object 
of scholarly and critical attention. Those who teach literature have a signifi- 
cant investment in classroom practice and outcomes; the general public joins 
us in significant concern over the choice of works to explore. In spite of this 
intense interest in what we teach and even how we teach it, however, until 
very recently, there has been less interest in the overall effectiveness of our 
efforts. We are more likely to discuss how we choose our texts than how we 
know whether or not we have achieved any of our teaching goals, be they con- 
servative, radical, or in between. Outcomes assessment would seem to be an 
obvious partner and resource for our work, and yet it often appears to many 
faculty members as fundamentally disconnected from the daily challenges of 
teaching and learning. Indeed, for many instructors in higher education, the 
most interesting question raised by outcomes assessment has been how to 
avoid getting stuck with it. Why? 

We propose a three-part answer: 1) assessment has been identified with 
government mandated accountability measures about which faculty and 
administrators have a range of reservations; 2) even when assessment is under- 
stood in more progressive terms as a means of improving student learning, it 
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has not been seen as a viable way of furthering the elusive and even ineffable 
heart of what goes on in literature classrooms; and 3) the best-known assess- 
ment efforts have targeted overall institutional performance and general edu- 
cation outcomes rather than the concerns and outcomes of specific disciplines. 
In what follows, we take up these issues. 

Assessment for Accountability vs. Assessment for Learning 

The current landscape for assessment in the United States takes its con- 
tours from the assessment movement that has developed over the last three 
decades, and even a cursory glance at this history suggests how constant the 
central debates of that movement have been. There has been forward move- 
ment in the field, to be sure, as — for example — increasingly sophisticated 
assessment instruments have been developed in response to increasingly 
sophisticated understandings of student learning and organizational develop- 
ment (Ewell, “To Capture”). Since its inception, however, the field has been 
bedeviled by tensions between assessment conducted for the purposes of 
accountability and assessment conducted for the purposes of improving teach- 
ing and learning (Ewell, “Emerging Scholarship” 9). 

Peter Ewell, who has been described as the “dean of the outcomes assess- 
ment movement in higher education” (Banta, Building a Scholarship 1), is also its 
premier — and, so far, its only — historian. Ewell has charted and reflected on 
the movement’s history in a series of essays (“To Capture,” “Emerging 
Scholarship,” “Perpetual Movement”), tracing its roots to research and prac- 
tice in four major areas that have all had an impact on the work of assess- 
ment: on student learning (a research history we’ll come back to); on student 
retention; on “the rise of program evaluation” and the development of “scien- 
tific management” approaches to education; and on the development of the 
“mastery and competency-based learning movement,” which is behind cur- 
rent assessment methods that focus on “evaluating student portfolios and other 
authentic evidence of student attainment.” The first of these bodies of work 
begins as early as the 1930s, and the others all emerging in the 1960s and later 
(“Emerging Scholarship” 4-6). 

While it has deep roots, then, the assessment movement picked up notice- 
able momentum in the mid-1980s. Ewell dates “the birth of the assessment 
movement” to the “First National Conference on Assessment in Higher 
Education, held in Columbia, South Carolina, in the fall of 1985” (“Emerging 
Scholarship” 7) and his choice of a conference as a key moment in the move- 
ment’s history is telling. For Ewell (and for us) assessment is a dynamic and 
inherently collaborative activity, and the work of assessment proceeds most 
effectively when it is draws on the community of those who have a stake in 
improving student learning. Feeding into and out from that meeting were a 
series of reports written from two major perspectives. On the one hand were 
those speaking on behalf of the academy (from, for example, the Study Group 
on the Conditions of Excellence in Higher Education, the Association of 
American Colleges, and the National Endowment for Humanities) and calling 
for the development of more “coherent” curricula, more pedagogical strategies 
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that sought to improve student achievement through “active learning” and 
“prompt and useful feedback,” and assessment of all of these efforts. On the 
other hand came “state-based calls for greater accountability,” from, for exam- 
ple, the National Governor’s Association (“Emerging Scholarship” 7-8). 2 

The tension between academic voices calling for educational reform on 
the one hand and government voices calling for accountability on the other 
continues to be manifested in every aspect of assessment work, from its moti- 
vation through its implementation to its use. That said, this tension is often not 
clearly understood. Thus, as Ewell also points out, the very word “assessment” 
can lead to confusion, as people use this blanket term to mean very different 
things: an effort to measure “an individual’s mastery of complex abilities,” or 
an effort “to benchmark school and district performance in the name of 
accountability,” or even “a special kind of program evaluation, whose purpose 
was to gather evidence to improve curricula and pedagogy” (“Emerging 
Scholarship” 9). Tensions and ambiguities notwithstanding, the assessment 
movement has persisted, and while the two fundamental lines of work — 
assessment for accountability and (to use Peggy Maki’s phrase) assessment for 
learning — have developed considerably over the last twenty-five years or so, 
both also still draw considerable critical commentary. 

On the accountability side, the reception of assessment has been shaped 
by some arguably unfortunate associations: many at first assumed it was, at 
best, a passing trend that would go away on its own (like the “sub-plot within 
assessment” that was Total Quality Management [“Perpetual Movement” 2]), 
and at worst an extension of No Child Left Behind (NGLB). Given that 
enthusiasm for accountability has only increased in the Obama administra- 
tion, as evidenced by the appointment of Arne Duncan as Secretary of 
Education and his accountability-oriented policies, the historical link of out- 
comes assessment to the Bush-era NGLB seems unshakable. Real as that con- 
nection may be, however, it is also — as we will later argue — limited in both its 
explanatory and evaluative power. Nevertheless, during the Bush administra- 
tion, accrediting organizations stepped up requirements for outcomes assess- 
ment in response to the 2006 report of the Commission on the Future of 
Higher Education, written under the authority of then-Secretary of 
Education Margaret Spellings. “A Test of Leadership: Charting the Future of 
U.S. Higher Education” (United States Dept, of Education) — or the Spellings 
Report, as it has come to be called — argued that universities have not been 
sufficiently accountable to their federal funding agencies and other stakehold- 
ers (parents, taxpayers, future employers), particularly in the areas of student 
learning and access. 3 Thus outcomes assessment came to many institutions as 
a top-down demand for accountability that shared its timing, its administra- 
tive origin, and apparently also its ideological impulse, with the notorious 
NGLB initiative. 

Outcomes assessment as practiced in higher education, however, has clear 
differences from NGLB: unlike NGLB, accrediting agencies have not mandat- 
ed standardized tests; they have for the most part allowed institutions to devel- 
op their own processes. There is no “failing school” designation of the sort 
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that has proven so unproductive with NCLB. 4 The accountability movement 
in higher education is not even the result of a federal mandate: accrediting 
agencies are not governmental bodies like the Department of Education, but 
rather private agencies funded by the institutions they serve. That said, the 
accrediting agencies have been influenced by federal agencies and state gov- 
ernments, as mentioned earlier, and showed interest in assessment results long 
before the Spellings Commission produced its report. Thus assessment has 
registered with many faculty members as a potentially reductive accountability 
effort that will replicate the problems associated with NCLB, and this percep- 
tion has to a high degree shaped its reception on college and university cam- 
puses. These negative associations, combined with the fact that faculty mem- 
bers often experience assessment as an unsupported and unsustainable addi- 
tion to their workload (the phrase “unfunded mandate” will inevitably come 
up), an intrusion into painstakingly developed pedagogical strategies, or even a 
violation of academic freedom, mean that it has not struck most instructors as 
an intellectually promising avenue. 

Skepticism about assessment for accountability has been matched to 
some degree by skepticism about — or perhaps more accurately, lack of 
interest in or even familiarity with — assessment for learning. As mentioned 
earlier, Ewell calls attention to a long line of research focused on generating 
information that can shape efforts to improve student learning (rather than 
hold institutions accountable) (“Emerging Scholarship”). Indeed, inquiries 
into student learning in the fields of educational and developmental psy- 
chology from the 1930s through the 1970s produced “basic taxonomies of 
outcomes, models of student growth and development, and tools for 
research like cognitive examinations, longitudinal and cross-sectional sur- 
veys, and quasi-cxpcrimental designs,” all with the goal of the improvement 
of learning (“Emerging Scholarship” 4). This research seems to be less well- 
known than the accountability-focused work mentioned above, and in the 
field of literary study, even books devoted to teaching generally fail to mine 
this rich resource. 

Education sociologist Steven Brint argues that the progressive agenda for 
teaching and learning has been carried forward by the ongoing efforts of 
organizations such as the Association of American Colleges and Universities 
(AAC&U) and the Carnegie Foundation for the Advancement of Teaching 
(along with the foundations that have supported them). Brint also notes, how- 
ever, that — while the teaching reform movement has advocated “active learn- 
ing experiences, commitment to diversity and civic engagement” alongside 
“challenging academic standards” — it has ultimately proven, in his view, “no 
match for the consumerism and utilitarianism of college student life” (4). 
Whatever the reason, our experiences have tended to confirm Brint’s observa- 
tion that commitments to assessment for learning and the information that 
such projects have produced tend to be overlooked in discussions of assessment. 

Many researchers actively involved in studying student learning today are 
consciously trying to retain the idealism of the reform movement and its focus 
on student learning, even as they look for ways to help institutions meet the 
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demands of accountability. This convergence of agendas has been increasing- 
ly evident in a range of major studies and initiatives. For example: 

• the Wabash National Study, which began as “a large-scale, longitudi- 
nal study to investigate critical factors that affect the outcomes of a 
liberal arts education” (Center of Inquiry in the Liberal Arts, 

“Wabash National Study 2006-2009 Overview”) now focuses particu- 
larly on helping “institutions to use evidence to identify an area of 
student learning or experience that they wish to improve, and then to 
create, implement, and assess changes designed to improve those 
areas” (Center of Inquiry in the Liberal Arts, “Wabash Study 2010”). 

• the National Survey of Student Engagement runs an Institute for 
Effective Practice, which aims “to draw upon current and future 
research initiatives to help faculty, administrators, and governing 
board members effectively link information about student experiences 
to efforts to improve academic programs and support services” (NSSE 
Institute for Effective Educational Practice); 

• the Higher Education Research Institute (HERI) helps individuals 
and institutions use assessment data in their ongoing work and as the 
foundation for a series of summer institutes: one on “campus decision 
making” (among other things), another on “problem solving using 
assessment to improve diverse learning environments,” and still 
another on “statistical methods that will enhance critical research,” 
“identify social and institutional inequities,” and “frame and design 
studies that ... support social justice and policy change” (DeAngelo); 

• CLA in the Classroom, a very successful spin-off of the Collegiate 
Learning Assessment (CLA), which uses “performance tasks” to assess 
students’ higher order thinking skills (Performance tasks provide stu- 
dents with materials to solve real-world problems.) (CLA in the 
Classroom); 

• AAC&U Valid Assessment of Learning in Undergraduate Education 
(VALUE) project begins with the premise that “to achieve a high- 
quality education for all students, valid assessment data are needed to 
guide planning, teaching, and improvement” (AAC&U). 

Even with the best of intentions and with an emphasis on the benefits to 
learning (while fulfilling the demands of accountability as well), assessment 
projects still pose multiple challenges. To reach the point of using evidence to 
improve student learning, we have to be convinced that the evidence at hand 
is measuring what we want it to measure, and here, perhaps, is the real issue 
for many faculty members: can we measure the learning outcomes that we 
value the most? This question points to one of the most enduring critiques of 
assessment, and it focuses on an issue that arguably pertains to many disci- 
plines even as it seems to have special relevance to the field of literary study: 
what Peter Ewell has labeled the “ineffability debate” (“Emerging 
Scholarship”). 
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In its most extreme formulation, the “ineffability debate” questions “the 
extent to which educational outcomes can be specified and measured at all” 
(“Emerging Scholarship” 17-18). Critics have wondered whether “ any attempt 
to look at outcomes directly was both demeaning and doomed to failure,” even 
as they have been put off by the fact that “assessment’s principal vocabulary 
appeared confined to education and the social sciences,” and have “character- 
ized as ‘positivist’ and excessively mechanistic” both “the rhetoric and the 
implied methods advanced by the assessment movement” (“Emerging 
Scholarship” 17). Ewell astutely observes that 

this classic complex of faculty reservations about assessment 
reveals some quite different underlying issues. Some are legiti- 
mately methodological, including appropriate reservations about 
the ability of off-the-shelf instruments and forced-choice methods 
to fully reflect collegiate learning, or fears about “teaching to the 
test.” Some are profoundly philosophical, based on a recognition 
that deep learning is always holistic, reflective, and socially con- 
structed. Still others are predominantly political, derived from 
faculty fears about loss of autonomy and creeping management 
control, as well as concerns about external intrusion into the cur- 
riculum. (17) 

For Ewell, the debate itself “has proven helpful in deepening assessment 
scholarship.” It has “forced practitioners to sharpen the philosophical ground- 
ing of the movement — rooting it in the tenets of scholarship and the process 
of teaching and learning,” has “reemphasized that the evidence used by 
assessment must always rest upon a peer-based community of judgment,” and 
has “forced explicit recognition of the fact that evidence is consistently con- 
strained by the context in which it is generated” (“Emerging Scholarship” 1 8). 

Ewell concludes that “[ejpistemological issues of this kind thus remain at 
the heart of the movement and remain healthily and vigorously contested” 
(“Emerging Scholarship” 18), and he is right. This collection of essays attests 
to the centrality of the “ineffability debate” in discussions of assessment, and 
if this collection can engage that debate and move it forward — looking espe- 
cially at its relevance for the discipline of literary study — it will have accom- 
plished one of its central aims. Certainly assessment raises a host of theoreti- 
cal and practical issues for all faculty, ranging from the concerns about 
accountability discussed above, through questions about how to balance 
already overburdened schedules and make strategic use of precious resources, 
to worries about the legitimacy of the work, the seeming redundancy of the 
data (after all, professors gives grades already), and the potential for inappro- 
priate use of outcomes results in promotion reviews or in other ways. But 
again, for those who teach literature, the most common problem seems to be 
the challenge of measurement itself. Many teaching in literature programs 
feel that while the mastery of calculus, for example, can be measured with an 
exam, learning in literary studies has an ineffable quality. A few peripheral 
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skills, the argument goes, might be measurable. We could probably figure out, 
for example, how many students understand proper citation or verb agree- 
ment, but doesn’t measuring these skills reduce the field of literature and cul- 
ture to instrumentality? Further, if we concentrate on measuring only what we 
think can be measured, we will leave behind the parts of the discipline that we 
find most truly valuable. Great teachers introduce their students to texts whose 
meanings are so many and so subtle, whose forms are so complex and irre- 
ducible, that any effort to codify the experience of reading and interpreting 
them seems to promise inevitable reduction. How can we measure the out- 
comes of such efforts, or even fully understand the efforts themselves? 
Contributors to this collection engage this and related questions from a variety 
of perspectives. 

The Need for Discipline-Based Assessment 

Returning to our opening question about why teachers of literature have 
been relatively reluctant to engage assessment as an intellectual and profes- 
sional issue, we want to suggest one additional and less recognized factor: the 
importance of grounding learning outcomes assessment in the disciplines. 
While sophisticated assessment projects generate a great deal of evidence 
about students and what they are learning, less cross-fertilization than one 
would expect has taken place between assessment for learning and efforts to 
define and achieve disciplinary goals. We suspect that this is in good part 
because so much research into student learning applies to institution-wide 
rather than disciplinary concerns, something that limits faculty investments 
and ownership of the process. 5 Nationally normed instruments such as the 
National Survey of Student Engagement, the Collegiate Learning Assessment, 
and Your First College Year, to name just three, provide various kinds of valu- 
able data that can help faculty and administrators understand where their 
policies and practices can be more effective in improving student learning 
overall. Nevertheless, teaching and learning also raise discipline-specific prob- 
lems and possibilities that these instruments cannot address, and it is our con- 
tention that assessment that does not take disciplinarity into consideration will 
inevitably run up against certain limits. 

Probably most of you inspired to read this volume agree: in the ADE sur- 
vey cited earlier, 78% of respondents felt that assessing a literature program 
raises particular issues not faced by assessment in other disciplines (“ADE 
Survey”). In pointing this out, we are not trying to make the case that literary 
study is exceptional, but rather that disciplinary focus is crucial in higher edu- 
cation; we imagine that responses would be similar in the history department 
and in the math department. While instructors in higher education often share 
some broad goals for their students, the subject matter itself nevertheless 
demands different strategies for thinking about learning. Further, the more 
general and sometimes generic assessment practices, we fear, can end up dis- 
couraging broadly based faculty participation. 6 If assessment strategies seem 
alien to disciplinary cultures, and if assessment practice itself comes across as 
a distinct field with its own aims and standards, then faculty are more likely to 
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continue to understand assessment as a process done to them from the outside 
rather than emerging from within disciplinary practice. Thinking about assess- 
ment as its own distinct field, unembedded in disciplinary practice, can defeat 
the purpose in several ways: at one extreme, it can tempt institutions to simply 
hire professionals to take care of an annoying accountability problem, or 
worse, it can impose goals and methods that faculty don’t understand as 
aligned with the purpose of their discipline. It can alienate the people most 
crucial to the process: the faculty. While we have much to learn, then, from 
scholars who have developed assessment strategies and considered their stakes, 
we also want to maintain the importance of disciplinary distinctiveness. 

Over time, we hope that new strategies for assessment will emerge that 
take into account disciplinary differences and, ideally, that use the tools of 
the disciplines themselves to engage assessment projects. Discipline-specific 
assessment methods and strategies certainly exist or are being developed 
across a number of fields: the National Institute of Learning Outcomes 
Assessment (NILOA) currently posts resources for nineteen disciplines on 
its website. The different concerns that can drive assessment in the disci- 
plines have also been subjects of study. Trudy Banta’s Assessing Student 
Learning in the Disciplines gathers together essays that explore questions of 
disciplinary assessment from multiple perspectives: what it means to assess 
disciplinary learning from the perspectives of faculty development and stu- 
dent learning; what is involved in assessing learning in disciplines subject 
to accreditation (such as nursing) and those not subject to accreditation 
(history and urban studies are both discussed); what is at stake in using 
“standardized tests in assessing learning in the major” (for example, the 
Educational Testing Service’s Major Field Tests) which — she notes — “can- 
not cover all the aspects of college students’ learning in a given major,” 
but “must be supplemented by other measures designed or selected by 
departmental faculty” (5-6). There is also very good work being done to 
link disciplinary learning to the broader goals of liberal education. 7 That 
said, there is much yet to do. 

In this rudimentary foray, we have concentrated on the stakes more 
than the strategies, as this seems to us to be the more urgent question. 
Essays in this volume certainly offer practical advice and explore some 
models of assessment projects, but for the most part, contributors here con- 
centrate on the intellectual and political meanings, possibilities, and limita- 
tions that emerge at the contentious crossroads of outcomes assessment and 
literary study. We aim to open up a two-way street: in one direction, educa- 
tors need to develop assessment strategies that reflect their own thinking 
about the purpose of their discipline. In the other direction, and as we 
hope will be in evidence in this volume, we can also use disciplinary strate- 
gies to think about the project of assessment itself and its place in higher 
education. We hope that faculty members come to see themselves not as 
passive executors (or resistors) of assessment directives, but rather as critical 
thinkers in shaping student learning on one side and assessment strategies 
and policies on the other. 
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The Essays 

In keeping with our hypothesis that the tools of the discipline are the best 
ones to exploit in assessing the learning intended to take place, we have 
focused on the literary figuration of the sublime, which is above all a way of 
trying to understand the ineffable, as a way of grounding this collection of 
essays. We want to know whether there really is a genuinely inarticulablc and 
so unmeasurable set of learning outcomes associated with literary study and 
we want to understand more fully the satisfaction, challenges, curious possibili- 
ties, frustrations, and sometimes outrages that various faculty members have 
experienced in the advent of assessment on their campuses. Our contributors 
are generally not assessment “experts,’' but rather reflective practitioners and 
insightful theorists who have brought their considerable critical skills to bear 
on the looming questions that assessment raises. These essays address good 
experiences and bad ones; our authors include assessment enthusiasts, assess- 
ment skeptics, and many somewhere in between. 

The collection opens with a series of four essays that think broadly as well 
as deeply about undergraduate student learning: about the goals of an under- 
graduate education, the value of disciplinary learning and especially literary 
study within the framework of that education, and the importance of knowing 
both how to assess what students are learning and then — on the basis of that 
evidence — how to intervene to bring their learning to a still higher level. Carol 
Geary Schneider frames the large questions of the discussion, considering 
what it means to offer students a liberal education — that is, an education 
focused on the development of broad cognitive and personal capacities, with 
grounding in disciplinary learning — in a society committed to the education of 
all of its citizens. She speaks in part from her position as president of a nation- 
al organization devoted to precisely this form of education to insist on the fea- 
sibility and, indeed, the necessity of defining even seemingly elusive goals 
clearly, finding ways to help students achieve them, and relying on evidence to 
attest to the work one has accomplished as well as what remains to be done. 
Against that strong national call Schneider positions the personal story that 
also grounds her argument and her work on the national scene. Rachclle L. 
Brooks moves directly into a discussion of how one might go about assessing 
the kind of learning Schneider discusses, arguing for the importance of 
attending not only to general education goals and outcomes, but to discipli- 
nary learning as well. Her distinction between what she calls the assessment of 
a discipline’s distinct learning outcomes and assessment within a discipline to 
determine how it contributes to the overall outcomes of a college education is 
especially valuable to those of us trying to do this work. Rosemary G. Feal, 
David Laurence, and Stephen Olsen report on the efforts of the Modern 
Language Association — the primary professional association for many teachers 
of literature — to engage with questions around the definition and assessment 
of student learning. And in the final essay of this section, Michael Holquist 
offers a rich context for current discussions of assessment, which he situates in 
relation to the history of universities as they have developed both in Europe 
and North America. His argument pays particular attention to Immanuel 
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Kant’s understanding of the university as a place that seeks to develop in stu- 
dents a capacity for “critical judgment” or “criticism,” even as it moves on to 
a deeply felt and powerfully articulated argument against the forms of stan- 
dardized assessment that he sees taking root abroad and, he fears, in the 
United States as well. 

From this opening set of essays, the collection moves to a second set that 
confronts questions of the ineffable head on, using the tools of literary 
study — and especially concepts of the sublime, which (again) we understand 
as efforts to theorize “ineffability” — to think through both the kind of learn- 
ing that characterizes literary study and how we might gain evidence of its 
accomplishment. W. Robert Connor grounds this set of inquiries with a 
study of Longinus’ seminal treatise on the sublime, reading it as an “educa- 
tional theory” in which the sublime emerges as a technology that “has pro- 
foundly enriching and liberating effects on the individual” and that — far 
from being ineffable — can in fact be “recognized and analyzed” (97). Donna 
Heiland advances a related argument, bringing the psychologist Mihalyi 
Csikszentmihalyi’s concept of “flow” together with eighteenth-century think- 
ing about the sublime to argue for the possibility of constructing classroom 
experiences that engage students intensely, and that, at their best, create the 
conditions in which students come to those moments of insight, the “aha” 
moments, that can seem so elusive in their workings. Sarah Webster 
Goodwin engages questions akin to Heiland’s but by another route, calling 
for the use of “our discipline’s most complex hermeneutics” (134) — its think- 
ing about the sublime and related questions — to assess students’ capacity for 
pathbreaking intellectual work, and arguing for the value of rubrics as “not 
a constraint but instead ... a means of liberation” in achieving this end 
(135). Lucinda Cole adds a valuable perspective to these essays’ discussions 
of sublimity and creativity when she makes a case for reclaiming the aesthet- 
ic — and within that frame, creativity — as a specific value in literary study, 
and for understanding creativity not as an individual talent but as a capacity 
cultivated by a socio-cultural system. Charles Alticri makes an equally strong 
case for the value of the aesthetic in both literature and literary study, but 
from a different angle. Focusing our attention on the particular values of the 
work of art and what it takes — and means — to appreciate that particularity, 
he demonstrates his point with a finely nuanccd reading of two key lines in 
Shakespeare’s Antony and, Cleopatra. And while he does not dismiss the possi- 
bility of developing a mode of assessment adequate to the complexity of 
such critical reading, he argues that we would do better to use available 
resources for the support of small classes that could effectively foster the 
development of such reading practices, rather than diverting them to a dis- 
tinct effort of assessment. 

The third group of essays come together around questions of “Politics, 
Institutions, and Disciplinary Goals.” Laura J. Rosenthal argues that out- 
comes assessment, in spite of its complications and limitations, might actu- 
ally benefit not just students but the discipline of literary study itself, which 
has come under attack from both the left and the right for its lack of a 
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clearly articulated purpose. She places the call to assess in the context of 
other considerable pressures on literary study in higher education and 
argues that assessment could become a form of advocacy by concentrating 
attention on our primary interface with the public: the classroom. Charles 
M. Tung proposes a new strategy for reading practice in response to the 
“coroner’s report” (199) that keeps getting written for literature depart- 
ments. He too argues that the classroom is the place to rethink the disci- 
pline. Negotiating between calls for “distant reading” (199) and a return to 
poetics, Tung explores the advantages of what he calls “thick reading built 
on the curricularizcd classroom” (207). “Although the “S” word has become 
virtually synonymous among academics with pedagogical sterility,” write 
Gerald Graff and Cathy Birkenstcin, “we want to make a progressive case 
for educational standardization by pointing out its unappreciated democrat- 
ic potential” (2 1 7). Graff and Birkenstein go on to reveal the limits of the 
anti-standardization argument and suggest serious, although not uncritical, 
attention to the findings of the “Spellings Report.” In a wide-ranging and 
ambitious essay, David Mazclla integrates his own experience launching the 
Center for Teaching Excellence at the University of Houston with both 
assessment and broader issues of institutional organization. Mazclla shows 
the indispensable value of “double-loop learning” at every level, from the 
day-to-day work of learning to institutional organization itself. He sees a 
vigorous engagement with the challenge of assessment as the best way to 
avoid some of the current challenges to institutional autonomy. 

The fourth group consists of “Case Studies and Templates.” In their 
essay “From Skepticism to Measured Enthusiasm,” Kirsten T. Saxton and 
Ajuan Maria Mance chart their “shifting attitudes as English professors, 
from trepidation to measured enthusiasm, for the value of assessment in our 
field of literary study” (259). Saxton describes the challenging process of 
revising her department’s “Introduction to Literature” course upon the 
painful (but possibly familiar) recognition that students could take more than 
one section without recognizing that they were taking the same course. In 
the second half of the essay, Ajuan Maria Mance reflects on the ways in 
which assessment practices could help insure that an institutional commit- 
ment to diversity gets translated into the personal and intellectual transfor- 
mation of students. Pat Belanoff and Tina Good offer a disturbing and cau- 
tionary tale in their efforts to prioritize assessment for student learning over 
increasingly bureaucratic forms of accountability. In this vivid account, 
Belanoff and Good emphasize the tremendous stakes involved and the 
urgent need for both individual and collective action on the part of faculty 
members. Jenny Bergeron and Russell A. Berman argue for the centrality of 
“critical reading” of literary texts in a foreign language to undergraduate 
education in the liberal arts, noting its value for cultivating not only stu- 
dents’ language skills, but also their capacity for sophisticated textual analy- 
sis, “cross-cultural hermeneutics,” and overall “intellectual maturity” (310). 
Noting that most curricula in this area are less focused on what students 
really need to develop this skill, they suggest ways to rethink what is taught, 
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why it is taught, and how it is assessed. In an essay that focuses specifically 
on the question of how language learning (in this case, Spanish) and the 
skills demanded in literature courses relate to each other, Jose Ricardo-Osorio 
makes the case that assessment can help us understand the continuity 
between those two kinds of learning. Finally, Barbara E. Walvoord — who 
has written and consulted widely on assessment issues — considers the chal- 
lenge of devising an assessment strategy particular to literature courses and 
departments. She offers a lucid, step-by-step guide to devising an increasing- 
ly specific rubric that captures key learning goals in a literature course. 

The essays in this volume, then, range widely in their views and approach- 
es to the project of assessment as it relates to literary study; nevertheless, they 
consistently combine theoretical insight and practical wisdom. There will be 
more to say on this subject, we are sure, and looking beyond our own field, we 
hope that scholars and teachers in other disciplines will further deepen our 
understanding of what is at stake in the work of assessment, too, by taking on 
this issue in the context of their own teaching and research. 


NOTES 

1 Ewell makes a similar observation when he comments on “widespread 
perceptions of assessment as an ‘administrative’ rather than an academic 
pursuit” (“Perpetual Movement” 3). 

~ The Study Group on the Conditions of Excellence in American Higher 
Education’s Involvement in Learning came out in 1984 (United States Natl. 
Inst, of Education); The Association of American College’s Integrity in the 
College Curriculum appeared in 1985; William J. Bennett’s To Reclaim a Legacy 
appeared in 1984, published by the National Endowment for the 
Humanities (of which Bennett was then Chair); and the National 
Governors’ Association’s Time for Results appeared in 1986 (“Emerging 
Scholarship” 7-8). 

^ The full text of the report is available on the United States’ Department 
of Education website. 

^ The full text of NCLB, which passed in 2002, is available on the US 

Department of Education website. For an example of the kind of debate 
inspired by the “failing schools” designation, see Schemo. 

^ It is instructive to know that — in the 1980s / 1990s, when assessment 
methods first began to shift to “naturalistic” assessment modes that took 
advantage of what institutions and individuals already did, and to “cur- 
riculum-embedded techniques” in particular — these efforts “first took 
place in the major field, where clear course progressions often existed 
and where faculty were better able to identify appropriate opportunities” 
(“To Capture” 106). These included using such existing assessment 
modes as exams, “capstone expcricnce[s] (e.g., a senior seminar, project, 
internship, or field placement)” or even less obtrusive assessment mecha- 
nisms, such as reviewing “syllabi and required exercises, together with 
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sample student products at different stages to roughly determine curricu- 
lar goal achievement” (106). Assessment of general education out- 
comes — which now dominate the national discussion — followed more 
slowly down this path (104). 

6 Wright makes this point as well, noting: “Current [assessment] offerings 
are overwhelmingly generic — that is, they do not speak specifically to the 
interests of a faculty member in sociology or geology, in art history or 
business. Yet the discipline is where a faculty member lives, literally and 
metaphorically. It is the discipline to which faculty members devote a life- 
times of scholarship. It is love for and expertise in the discipline that they 
hope to cultivate in their students” (7). 

^ See Brooks’ essay in this volume, which discusses efforts to assess broad 
liberal learning outcomes as they are achieved in the disciplines of classics 
and political science. The American Society for Biochemistry and 
Molecular Biology has also done work that is relevant here, exploring the 
extent to which the Society’s disciplinary learning goals map onto 
AAC&U’s liberal learning goals. 
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TRANSFORMATIVE LEARNING— MINE AND THEIRS 

CAROL GEARY SCHNEIDER 


THIS VOLUME OF ESSAYS ENGAGES two broad national dialogues, one 
concerned with assessment and its relation to learning, and a second con- 
cerned with the future of the liberal arts in general and the humanities in 
particular. After many years of working with both these issues, I am persuad- 
ed that progress on each will depend on a fundamental recasting of our most 
basic organizing assumptions about what actually counts as liberal learning. 
We need new frameworks to characterize the main goals of a twenty-first- 
century liberal education 1 — both to guide students in achieving a liberal and 
liberating education, and also, crucially, to develop meaningful assessments 
that speak to our most important educational goals. 

In the passages that follow, I share the personal history that shaped my 
views on transformative liberal education: what it entails and how we can fos- 
ter it for a far more inclusive array of students. And, in the end, I come back 
to the question that inspired this entire volume, the relationship between the 
assessable and the ineffable in the contexts of our students’ own hopes for a 
better future. 


It was 1976. Fresh from a highly specialized course of study in early mod- 
ern history, first at Mount Holyoke College and then as a graduate student at 
Harvard University, I had taken a position at Chicago State University. 
Chicago State was at the time in the midst of fast-paced change, both in mis- 
sion and in demographics. A former teachers college that was redefining itself 
as a regional comprehensive campus, it had become an overwhelmingly 
majority “minority” institution — albeit with many white faculty. 

Many of the students at Chicago State had come from dismally inade- 
quate public schools in Chicago — most as recent graduates and some, then 
returning to college in mid-life, many years earlier. Hired by the University 
for its Ford Foundation-launched “University Without Walls” (UWW) pro- 
gram and quickly assigned to deal with other innovative programs as well, I 
was working both with a cadre of very capable, but nonetheless still un-crcdentialcd 
“returning adult” learners and also with a set of notably under-prepared 
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traditional-age college students. 

I was attracted to both the challenges and the possibilities — the energy 
and native talent — that Chicago State students presented. Inspired like many 
of my age cohort by the civil rights momentum of the 1 960s, I actively want- 
ed to work with these first-generation college students. 

Alas, as I soon realized, I was completely lacking in practical knowledge 
about how to respond to our students’ educational needs. My graduate work had 
not included even an hour’s worth of time on the real-world students I might find 
in my classroom, much less on the mysterious subject of “learning.” Now, thanks 
to these suddenly glaring omissions, I found myself in uncharted waters. 

With the traditional-age students, I faced the fundamental question of 
what it means to foster “critical inquiry” in young people who in some cases 
had not yet mastered the paragraph. These students were appealing and ener- 
getic and hopeful about their futures. But many of them needed fundamental 
writing instruction at a level that I could barely envision, much less effectively 
provide. A well-crafted lecture — or even a seminar — on Renaissance history 
was not going to do the job for students who had been searingly underserved 
by their previous schooling. 

With the returning adult students, I faced a massive disconnect between 
their very practical reason for returning to college — to obtain a work-useful 
degree in the shortest possible amount of time — and my own commitment, 
supported by the mission of the UWW program, to ensure that their individu- 
alized plans of study added up to a “liberal” or “liberal arts” education. 

All the UWW students already were working because concurrent job 
experience related to their studies was a firm requirement for admission to the 
program. By design, UWW built out from their actual career interests, seeking 
individualized ways of bringing a liberal education context to their particular 
educational priorities. 

I loved the idea of helping students “individualize” their liberal education, 
which was, in fact, exactly what I had done as an undergraduate at Mount 
Holyoke. But very soon into my work at Chicago State I came to the uncom- 
fortable but world-altering realization that for all the years I had spent in some 
of the nation’s most admired “liberal arts” institutions, I had only the vaguest 
working conception of what I — or we in our UWW program — even meant by 
a “liberal” or “liberal arts” education (see fig. 1 for working definitions of 
these concepts). 

Even less had I developed a conception of liberal education that might 
create common ground with the interests, inclinations, and very basic practical 
needs of the Chicago State students I was employed to guide and teach. 

We in the UWW program were committed to providing a liberal educa- 
tion for our working adult students. Chicago State was committed to providing 
a well-rounded education, with a significant general education component, for 
all its students. But what exactly did that mean for our actual practice? And 
what did it mean for my guidance to students who shared neither my enthusi- 
asm for liberal education nor my presuppositions about what was important to 
a good education? 
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LIBERAL EDUCATION: An approach to college learning that empowers individuals and prepares them to deal 
with complexity, diversity, and change. It emphasizes broad knowledge of the wider world (e.g., science, culture, 
and society) as well as in-depth achievement in a specific field of interest. A liberal education helps students 
develop a sense of social responsibility as well as strong intellectual and practical skills that span all major 
fields of study (e.g., communication, analytical, and problem-solving skills), and includes a demonstrated abil- 
ity to apply knowledge and skills in real-world settings. 

LIBERAL ARTS: Specific disciplines (e.g., the humanities, sciences, and social sciences). 

LIBERAL ARTS COLLEGES: A particular institutional type — often small, often residential — that facilitates 
close interaction between faculty and students, while grounding its curriculum in the liberal arts disciplines. 

ARTES LIBERALES: Historically, the basis for the modern liberal arts: the quadrivium (arithmetic, geometry, 
astronomy, and music) and the trivium (grammar, logic, and rhetoric). 

GENERAL EDUCATION: The part of a liberal education curriculum shared by all students. It provides broad 
exposure to multiple disciplines and forms the basis for developing important intellectual, civic, and practical 
capacities. General education can take many forms, and increasingly includes both introductory and advanced 
levels of learning. 

Fig. 1 . “Guide to Frequently Confused Terms," adapted from Association of American Colleges and 
Universities, Greater Expectations: A New Vision for Learning as a National Goes to College (25). 


I knew, of course, that a liberal education certainly comprised critical 
thinking and writing, the skills so many of my traditional-age students conspic- 
uously lacked. (The adult students’ competence level was higher, in part 
because the admissions process for UWW required substantial writing.) But 
how did one cultivate “critical thinking” in students who were still struggling 
with the formulation of their thoughts in standard English? 

With both sets of students, I faced the question of the relationship 
between liberal education and a major in one of the arts and sciences disci- 
plines. Somewhat unthinkingly, I had absorbed from the DNA of my own 
educational experience the assumption that the transformative power of a lib- 
eral education was possible only in the arts and sciences disciplines and that a 
focus on some other field was a different species of learning altogether. 

But almost none of my students — either the returning adults or the tradi- 
tional-age students — intended to major in one of the arts and sciences. They 
were in college to expand their opportunities, and for most that meant studying 
a field whose title seemed plainly related to real-world jobs: business adminis- 
tration, marketing, criminal justice, education, the various health fields. 

What did it mean to provide a liberal education if students were studying 
“non-liberal arts” subjects? 

Given the students’ choice of major fields, the general education or distri- 
bution requirements appeared to loom large in the achievement of that “liber- 
al education” signature. And, in fact, it was a degree requirement at Chicago 
State that all students needed to complete at least thirty hours in the humani- 
ties, the social sciences, the sciences, the arts, mathematics, and writing. 

Perhaps, I mused, it was primarily general education — never seriously dis- 
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cussed at all, either in my undergraduate or graduate studies — that held the 
real key to a “liberal education.” This was an interesting possibility, especially 
in Chicago, where the weight of the University of Chicago’s much-celebrated 
“core curriculum” carried outsize influence, with faculty at least, even at a 
very different kind of institution. 

But general education at Chicago State — and almost everywhere else — 
was defined in terms of a menu of “distribution categories” rather than as a 
focused set of core courses. And distribution requirements soon proved, in 
practice, to be a very weak key indeed to a liberal education. In truth, many of 
my students seemed to regard the general education requirements not as the 
key to excellence, but rather as a mystifying or even infuriating impediment to 
their own intended purposes. 

One particularly memorable adult advisee, faced with my insistence on 
the “distribution requirements” that were written into the UWW program, 
angrily delivered her written decision to take two courses in American history 
“in order to fulfill the requirement that I pay money to study subjects in which 
I have no interest.” 

We were clear at Chicago State that she had to fulfill the arts and sciences 
requirements. She — heading for a career in elementary teaching! — was equal- 
ly clear that she could see no point to them. 

These conflicts, which were frequent, pointed to the larger question: What 
were our root goals in insisting on arts and sciences general education courses 
as a necessary component to the degree? 

And even more important, how would I transform students’ skepticism or 
outright resistance into something resembling the transformative learning I 
myself had experienced in college? 

And, finally, what exactly did we mean by liberal education? In what way 
did we expect liberal education to be a meaningful resource for these first gen- 
eration students? 

Fast Forward to the Present 

All these questions launched a journey of inquiry, self- reflection, and rich 
if decidedly “on-the-job” learning that led eventually to my current role as 
president of the Association of American Colleges and Universities (AAC&U). 
AAC&U is the major national association working on the quality of under- 
graduate college learning, and our mission is to make the “aims of liberal 
learning a vigorous and constant influence on institutional purpose and educa- 
tional practice in higher education” ( Strategic Plan). Founded in 1915 to support 
the primacy of colleges of arts and sciences — both independent colleges and 
units within larger universities — AAC&U has evolved over the past twenty 
years into a “big tent” association that draws its 1,200 institutional members 
from all parts of postsecondary education, private, public, large, small, two- 
year, and four-year. Collectively, AAC&U members have embraced a call to 
“Aim High — and Make Excellence Inclusive” ( Strategic Plan) by adapting the 
traditions and strengths of liberal education to twenty-first-century contexts 
and challenges. 
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Readers familiar with AAC&U’s current work on both the “aims” and 
“essential learning outcomes” (see fig. 2) of liberal education will see immedi- 
ately that all the questions I faced as an underprepared young academic three 
decades ago now stand directly at the heart of AAC&U’s current signature 
initiative, Liberal Education and America’s Promise (LEAP). 2 


LIBERAL EDUCATION: THE ESSENTIAL LEARNING OUTCOMES 

Knowledge of Human Cultures and the Physical and Natural World 

• Through study in the sciences and mathematics, social sciences, humanities, histories, languages, 
and the arts 

Focused by engagement with big questions, both contemporary and enduring 

Intellectual and Practical Skills, Including 

• Inquiry and analysis 

• Critical and creative thinking 

• Written and oral communication 

• Quantitative literacy 

• Information literacy 

• Teamwork and problem solving 

Practiced extensively , across the curriculum, in the context ot progressively more chailenging prob- 
lems, projects, and standards for performance 

Personal and Social Responsibility, Including 

• Civic knowledge and engagement — local and global 

• Intercultural knowledge and competence 

• Ethical reasoning and action 

• Foundations and skills for lifelong learning 

Anchored through active involvement with diverse communities and real-world challenges 

Integrative Learning, Including 

• Synthesis and advanced accomplishment across general and specialized studies 

Demonstrated through the application of knowledge, skills, and responsibilities to new settings and 
complex problems 

Fig. 2. “The Essential Learning Outcomes,” Association of American Colleges and Universities, College 
Learning for the New Global Century: A Report from the National Leadership Council for Liberal Education and 
America’s Promise (3). 


The LEAP initiative, launched in 2005 and designed to extend at least 
through 2015, explores for students and society alike the same issues I was so 
ill-equipped to address — either in theory or in practice — when I left graduate 
school and began to work with first-generation college students in all their 
variety, energy, hopefulness, and need. But LEAP, I hasten to say, benefits from 
three decades of work throughout higher education on how best to support 
first-generation students, and returning adults as well, as they come in ever 
larger numbers to higher education. 
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In 1976, 1 was tackling core questions of educational purpose and effec- 
tive practice largely on my own. Today, many networks of scholars and teach- 
ing faculty are working together, with a growing sense of shared mission, on 
just these questions. Collectively they are building a body of evidence on 
“what works” in higher education, especially with students from underserved 
communities. And, steadily if too slowly, reformers are beginning to wield sig- 
nificant educational influence across all sectors in postsecondary education. 

The truth is that my own experience at Chicago State, which seemed so sui 
generis at the time, was in fact a very small part of a much larger generational shift 
that was just starting in that era to gain momentum across higher education. 
Colleges and universities were opening their doors wider than ever before, self- 
consciously reaching out to embrace students and communities that traditionally 
had had very limited access or no access at all to higher education. Programs 
aimed at returning adults were multiplying all over the United States, and by the 
turn of the twenty-first century, both older students and underprepared tradition- 
al-age students would constitute an emerging majority in higher education. 

All of this has sent the academy “back to school,” so to speak, to better 
understand the practices that effectively foster students’ intellectual and person- 
al development, and to invent, test, and scale up the uses of what we now call 
“engaged learning” to better support this increasingly significant population. 

As a result, the educational practices AAC&U advances through LEAP 
build from the collective insights of innovative faculty and academic leaders 
throughout the United States — at “national institutions” and “regional institu- 
tions” alike. What is new, I believe, is the effort AAC&U is making to synthesize 
many different “reform agendas” — ranging from “writing across the curricu- 
lum” to diversity learning, community-based learning, undergraduate research, 
and the like — to provide a comprehensive “guiding vision” for higher educa- 
tion, for our students, and for the public at large. (See fig. 2 and note 2.) 

LEAP is not launching a new direction in higher education so much as 
turning a spotlight on far-reaching changes that have already begun to alter 
our understanding of both the aims and the signature practices of a twenty- 
first-century liberal education. These changes are intended to serve everyone, 
but they have particular “compensatory” benefits, it turns out, for higher edu- 
cation’s “new majority students” who are very much like the students I met so 
long ago at Chicago State (Kuh 17-20). Today’s students work while studying; 
they commute rather than live on campus; they bring the world to the class- 
room; they often are underprepared for college; and, on every level, they challenge 
the academy to rethink its most basic precepts about excellence and inclusion. 

Over time, if too slowly, these recently included students have forced revo- 
lutionary change both in the practice of liberal education, and now through 
the LEAP amplifier, in its stated purposes as well. These changes remain a 
work in progress, as all of us learn together how to better serve the millions of 
underprepared students who are now the new majority in American higher 
education. But, taken together, the myriad endeavors and experiments all over 
the United States to better serve contemporary college students from every 
walk of life are adding up both to a new vision for inclusive excellence in lib- 
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eral education and to a new era of more educationally intentional and inte- 
grative practices that can help our students make rich connections between 
what they learn in college and the lives they want to live. 

Fulfilling the larger promise of a liberal education forces us to think much 
harder — and to learn from many pioneering practitioners and scholars — about 
the connections between knowledge and its applications, and about the way 
experience figures in making knowledge both illuminating and empowering. 

In the pages that follow, I share some of my own “lessons learned’' from 
this shared journey of inquiry and reflection on the aims and practices of a 
contemporary liberal education. And, at the end, I come back to the question 
that still seems to haunt entirely too many academics: Is something essential 
actually lost when we seek to put liberal education to work in the wider world? 

As you will guess, my answer is no — not only is nothing lost, but much is 
gained. And, in the context of this particular set of essays, I would add that 
nothing is lost and much is gained when we actually hold ourselves (as faculty 
and institutional leaders) accountable for helping students show that they 
have achieved the explicit “outcomes” of liberal education. But as with all 
transformative insights, my answers are informed by my own experiential 
learning. As I believe is often the case with students, what I learned from 
experience helped me recognize the inadequacy of my initial assumptions 
about liberal learning — and also propelled me to enlarge them. 

My Own Transformative Learning 

Back to Chicago State in 1976 and beyond. Prompted both by populist 
idealism and by my increasingly urgent questions about the quality of student 
work, I signed up to take part in a voluntary “study group” whose members 
wanted to probe the usefulness of the emergent “competency movement” in 
higher education. Bringing together faculty from diverse institutions and a 
broad range of mostly liberal arts disciplines, together with a few administra- 
tors, the study group intended to assess whether there was anything of value in 
a competency-based approach to learning for underprepared college students. 

In my view, the competency movement of that era and the learning out- 
comes movement in our own time have much in common, most notably the 
emphasis on translating the “tacit knowledge” scholars unconsciously live by 
into “explicit standards” that can guide faculty and students alike in the devel- 
opment of intellectual and practical capabilities. Defining the expected out- 
comes or competencies students need to achieve and making standards for 
their performance explicit is one feature of this approach, but this is only part 
of the larger task. The real pay-off in an outcomes approach comes when fac- 
ulty learn how to develop sequenced assignments that enable students to prac- 
tice the intended learning and when students learn to take responsibility for 
meeting or exceeding the expected standards of performance. 

The working assumption behind this approach is that every field creates 
and privileges certain procedures for making an argument and/or testing a 
course of action, but that too often — especially in the arts and sciences — fac- 
ulty have internalized this “procedural knowledge” without actually ever nam- 
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ing it. As a result, faculty often are hindered in helping students grasp the ele- 
ments of what counts as good work and what does not. By studying examples 
of “good work’' and “poor work,” we can give voice to our own tacit stan- 
dards and, most important, provide clearer guidance and feedback to novice 
learners so that they can make progress toward higher levels of achievement. 

Ultimately, we hope, novice learners will move beyond trying to follow 
“rules of good practice” to become both insightful and inventive in their 
deployment of these frameworks. But making standards explicit provides the 
needed scaffolding to at least get new learners working in a productive direction. 

In effect, therefore, that informal study group became my first introduction 
to the whole idea of intentionality and explicit expectations as a framing “com- 
pass” that would help students navigate the mysteries of college-level work.-’ 
The group was useful in pointing me toward the literature of competency-based 
learning and toward examples of competency-based programs that already 
were springing up at different Chicago-area colleges and universities. 

What stands out for me now, however, from that 1976 effort to better 
respond to the needs of my actual students, is the vigorous debate over the 
very concept of “competencies” that broke out on the first evening that the 
study group convened. We were all curious about the entire idea of competen- 
cy-based learning. But many were far from convinced. 

“The most important forms of learning in the liberal arts arc ineffable!” 
contended one young faculty member who was then teaching in the Chicago 
community college system. I listened with interest since this scholar came from 
my own graduate program at Harvard. He had been working with underpre- 
pared graduates from the Chicago public schools somewhat longer than I. 

Had he found some way to sustain our ivy-infused values and teach his stu- 
dents successfully at the same time? 

“The whole idea of breaking history or literature down into ‘competen- 
cies’ is repugnant,” he continued with vigor. “It distorts and destroys the very 
things that are most valuable to our students!” “What we really need to do,” 
others agreed, “is help students see what is so exciting about our disciplines. 
We have to model, in our own teaching and style, the passion and excitement 
we ourselves feel for our subjects.” 

Having tried this strategy already, I was skeptical. And I was not alone. 

“We need to make our subjects accessible to the actual students we have,” 
protested others. “We can’t just tell them that the liberal arts are valuable for 
reasons that are beyond explanation. We have to somehow connect our sub- 
jects to the things the students themselves consider important. We have to 
break through our own mystique.” 

“Our students seek jobs and career advancement,” others chimed in. 
“How do we make the liberal arts relevant to those goals?” 

The debate raged on — in that forum and countless others like it — and still 
does today, in college and university contexts all across the nation. Even as I 
have been writing this essay, there has been a new national furor over whether 
the humanities are and should be “use-less,” or whether it is (finally) time to 
better explain and make good on their real-world significance. 
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For me, however, that informal study group clarified the core question at 
stake — demystification — and set me off on a course I have pursued for over 
three decades: the effort to break out of “my discipline” as a self-contained 
and self-referential scholarly community and to think much harder about the 
ways in which I hoped the insights of this discipline would actually be used by 
students in their lives beyond the classroom and beyond the academy. 

I was willing to concede, for the sake of argument, that some of the pleasures 
of a liberal education really are ineffable: personal, interior, spontaneous, exhila- 
rating. But those moments do not come at the beginning, and they are not likely to 
come at all if novice learners end up viewing the arts and sciences mainly as a set 
of barriers they need to get “out of the way” as soon as possible before proceeding 
to the “real point” of college. 

The challenge, I came to believe, is not one of “getting students excited” 
by how well we perform in their presence but, rather, of creating opportunities 
to help students see more in their own questions and explorations than they 
could have discovered on their own. Our task as educators, I now think, is to 
give our students a motivating reason to invest themselves in the project of 
developing those habits of mind and heart that characterize a liberally educat- 
ed person (see fig. 2). 

In this context, the skills developed through liberal education are a means to 
an end rather than ends in themselves. The ultimate goal is to foster students’ 
own growing capacity to work productively on projects and problems that they 
themselves want to address. 

Prompted by this insight, I began to reflect on how I myself had “gotten 
hooked” on scholarly inquiry and what light that might shed on how to help 
my students toward the illuminating power of a liberal education. 

The love of learning that a scholar gains (and uses to make a living) 
comes with an acquired ability to “make a discovery,” whether that discovery 
is one of flashing insight, new connections between elements previously seen 
as separate, or the pleasure that comes when painstaking work actually begins 
to take form and even significance. 

How had I developed that inclination myself? How, in particular, did I 
move from my high school standing as a young person willing to work 
extremely hard on subjects I hated, simply because I wanted a good grade, to 
someone who found learning illuminating and, in a profound sense, necessary? 

Miss Brock 

Those reflections, all prompted by my Chicago State students and that 
informal Chicago study group, led me back in turn to Miss Brock, the faculty 
member who taught both semesters of my required year-long “baby English” 
course at Mount Holyoke. Prior to my arrival at Chicago State, I had come to 
think of myself as a budding scholar, someone who wanted to contribute to 
the advancement of knowledge in early modern history. But now I reached 
back to my initial year of college, when I was not a professed future scholar at 
all, but rather a somewhat confused young woman who was trying to figure 
out what was expected of her at Mount Holyoke College. 
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I had my own “transformative experience” in that initial year of college, a 
set of catalytic events that made me genuinely interested in learning and the 
light it might cast — not on the world’s understanding and certainly not on the 
advancement of scholarship, but rather on my own actual decisions and choic- 
es. What were those experiences? Looking back, I thought with new attentive- 
ness about the role Miss Brock, my first year English professor, had played in 
that transformative experience and what her example might mean for the very 
different kind of students I was working with at Chicago State. 

Praised as a strong writer all the way through high school, I had not want- 
ed to take the two-semester required “baby English” course at all. To earn 
exemption, I had taken an optional reading and writing take-home examina- 
tion over the summer. (I failed.) Once enrolled, I did not find Miss Brock’s 
teaching particularly compelling. Fascinated with iconography and metaphysi- 
cal poetry, she often left me absolutely clueless as to what she was talking 
about. She had brilliant students with whom she was constantly in dialogue 
during class meetings. I was not one of them. 

Miss Brock put me forever in her debt, however, by the way she 
approached our research papers, which were required in both semesters of 
baby English. Painstakingly, she took the time to call in each one of her stu- 
dents for a separate discussion about the possible topic for our initial research. 
And in that conversation, she asked me to identify “a real question; something 
you really want to know the answer to; something whose answer is important 
to you.” 

Initially I was puzzled; this kind of exchange was completely new in my 
experience. But finally, a bit reluctantly, I told her my “big question.” What I 
most wanted to know, I explained, was whether miracles in fact “proved” that 
the Roman Catholic Church was really the One True Church, as I had been 
taught and raised to believe. 

To her enormous credit, Miss Brock did not laugh, or frown, or even allow 
a small sliver of a smile. (You, dear reader, may have done less well.) Instead, 
she nodded affirmatively; thought for a couple of minutes and then suggested 
that I would find it “profitable” to do my paper on mystical experience, both 
Christian and non-Christian. She suggested a couple of books to get me start- 
ed; she warmly encouraged me to look into some of the great mystical poets. 
She invited me to come back in to see her as my work progressed. 

And off I went, on what turned into one of the great investigations of my 
life. Or so, in my own mind, I considered it. In truth, as she probably surmised, 
I was reconsidering my relationship (privately) to my religious upbringing. The 
reading, thinking, and writing I did for that paper was transformative in myriad 
ways. It changed the way I thought about religion, my own choices, my own 
interests. It “hooked” me on scholarship as a source of insight and power. 

It was Mount Holyoke’s practice to retain copies of all the first-year papers 
students wrote in baby English and to return those collected papers to us when 
we graduated as seniors. When I received this collected set in my last week of 
college, I reached eagerly for that initial research paper on mystical experience. 

What I found was a paper that was stunningly naive in its framing and 


36 



TRANSFORMATIVE LEARNING— MINE AND THEIRS 


utterly hesitant in its conclusions, which were shyly advanced only in the final 
two pages. Mostly the paper was descriptive rather than analytical as I strug- 
gled to describe a different order of experience from anything I had previously 
encountered and to somehow connect these experiences to my own questions 
about the standing of Catholic doctrine. 

And yet, naive and novice though it was, this was the paper and this was 
the work that had launched my own commitment to scholarship — and more 
fundamentally, to the value and significance of a liberal education. 

The key, I am persuaded, was the original question — and especially the 
generosity with which my teacher received my original “big question.” I really 
wanted to know whether the Catholic Church was the One True Church, and 
the power of what was going on in my mind, in my own private thoughts as I 
approached this topic, vastly exceeded the actual sophistication of the written 
work itself. In my own mind, that investigation into a different realm of reli- 
gious experience was world-enhancing. And so, in truth, it was. 

Reflecting on all this, I began to put similar insights to work with my 
own students. In a variety of different ways, I set out to get them to tell me 
their own “significant questions” and I looked for ways to connect those 
questions with their assignments in my courses. I did not anticipate that any 
of them would choose a career in scholarship, but I did expect them to take 
seriously their own questions, and to put pen to paper in the exploration of 
those questions, using evidence, using sources, making an argument, devel- 
oping a case. 

Mount Holyoke was, of course, a small liberal arts college. The programs 
I led at Chicago State similarly were small programs, serving a few hundred 
students, rather than thousands. 

Is it feasible to imagine that this kind of strategy can be used on a broader 
scale, for example, in the general education courses that are required almost 
everywhere? 

One answer to this is that almost every college, university, and community 
college now teaches at least one writing-intensive course in a small-class context. 
Sometimes the class is basic composition; sometimes it comes in the form of a 
topically organized “inquiry” seminar. Some institutions also provide “first-year 
experiences” that include small-class workshops and seminars designed to teach 
students how to navigate the college or university environment. 

Just as Miss Brock used “baby English” at Mount Holyoke, cither context 
could be used to provide students with both opportunity and incentive to 
research their own “big questions” — and to discover, as I did, that scholarly 
work can make a difference in one’s own life, as well as in the life of our society. 

The most important challenge we face in fostering transformative learning 
is not the absence of contexts in which it could feasibly be attempted. Rather, 
it is our willingness to accept a state of affairs — endemic throughout higher 
education — in which some students benefit from the best that a liberal educa- 
tion can offer, while millions of others, typically students from less advantaged 
backgrounds, are steered toward narrow training and less ambitious goals for 
their own college learning. 
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So, Is Liberal Education “Ineffable?” And Can It Be Assessed? 

Clearly, something happened in my own experience at Mount Holyoke 
that was personal as well as powerful. And similarly, I believe, when that “light 
goes on” for our students, it goes on for an individual, helping that particular 
student make sense of his or her own very particular questions, interests, con- 
cerns, or commitments. 

But knowing how personal a liberating experience really is not does release 
us from a foundational responsibility. Through guided practice, feedback, and 
assistance, we still need to help students develop the analytical and investigative 
tools they need in order to get beyond the initial question — however naive or 
instrumental or relentlessly “practical” it may be — and to explore the implica- 
tions of their question, to develop an argument, to use evidence, to consider 
the alternatives, and to be able to explain what they have concluded and why. 

As my vignette makes clear, Miss Brock had three goals in mind (at least) 
for my assignments. The first was that I should care about the work I was 
doing, and take it seriously. The second was that I should expand beyond the 
boundaries of my initial knowledge and experience. Hearing my question, she 
did not hesitate to send me off on a topic that I did not even know existed — 
the realm of mystical experience, non-Christian as well as Christian. 

Finally, she expected me to write a documented research paper, and to 
show that I was using my sources in answering my question. When I handed 
in the paper, she undoubtedly saw — as I saw, four years later — that my first 
research paper was written in very broad strokes, long on description, short on 
evaluative interpretation. When we met during the second semester about my 
second research paper, she suggested that I write on the poetry of Gerard 
Manley Hopkins. At the time, I thought she was pointing me toward a poet 
who himself was struggling with religious doubt and passion, a point of refer- 
ence for my own implicit quest. 

Hopkins’ religious struggles undoubtedly were the reason Miss Brock 
chose that particular poet. But now, looking back over the years, I suspect that 
she also wanted me to work in a more focused way on “close reading” of care- 
fully constructed poetic texts. In that second paper, far more than in the first, I 
began to ground my claims in textual analysis; to read at a deeper level; to 
develop an argument that took seriously the words and the textual structure so 
carefully created by the artist. 

In other words, still honoring my own significant questions and commit- 
ments, Miss Brock strategically linked those questions to the intellectual tasks 
of both “close reading” and evidence-based argument. What she gave me was 
the best of two worlds: serious attention to my own questions, which were any- 
thing but “scholarly,” and serious guided practice in developing much better 
analytical skills than I had brought with me to college. 

This distinction can help us, I believe, differentiate between those aspects 
of liberal education that really are “ineffable” — or at least highly interior and 
personal — and those that lend themselves directly to meaningful assessment. 
The assessment questions pertain not to the conclusion I reached in my explo- 
ration of religious and mystical experience, but rather to whether I was able to 
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examine different positions on my topic, consider evidence in support of those 
positions, and develop an evidence-based position of my own. I did not, in 
fact, do that particularly well in my first research paper for Miss Brock. But, by 
the end of my initial year in college, I had acquired the concept of “close 
reading” and had started to develop some facility in connecting my argument 
to textual evidence. The skills required to make an evidence-based argument 
can — and should — be both practiced and assessed not once, but again and 
again, from early courses to culminating capstone experiences. The focus on 
constant practice, at increasingly higher levels of expectation and achieve- 
ment, is a necessary part of an empowering liberal education. 

I was fortunate that “baby English” was a two-semester course, taught by 
the same instructor each time. She saw where I needed to improve, given my 
work in the first semester, and she deliberately gave me assignments that 
would help me improve in the second semester. 

For today’s students, electronic technology offers new ways to achieve the 
same kind of effect. As colleges and universities begin to adopt electronic port- 
folios to capture student work, it becomes possible for any faculty member to 
peruse a student’s written work and discern whether that student is actually 
making progress in the development of analytical and interpretive skills, or 
just repeating the same weak techniques again and again. It becomes possi- 
ble — at least potentially — to give students assignments that build from their 
strengths and help them correct their weaknesses. In principle, we could do 
today, for all students, what Miss Brock did so brilliantly for me. But to make 
full use of this potential, we would have to move beyond the idea that the pro- 
fessor’s task is to profess. We would have to insist, instead, that the professor’s 
real task is to assess where students currently are in their development of key 
intellectual capacities and, building from that assessment, to help students 
move to a higher level of both effort and achievement. 

Similarly, electronic technology gives us the power to see whether students 
seem to pursue particular “big questions” or issues over time, or whether they 
are just producing assignments because they have to. Is the student’s learning 
starting to “come together” in transformative ways, across different courses 
and experiences? Does her portfolio show that she is probing particular issues 
and themes? Making connections across both courses and disciplines, and 
between her lived experiences and her studies? If yes, what catalysts might 
take the student’s integration of learning to an even higher level? If not, what 
is standing in the way? How can the student’s academic work become a more 
powerful resource, both for integrative learning and for the goals that brought 
her to college in the first place? 

The students’ own work over time provides insights into all these ques- 
tions. We need to take seriously the evidence of that work, and, as faculty, pro- 
vide both guidance and structure to help students take their capacities — and 
their questions — to a higher and fuller level. 

Liberal Education and America’s Promise 

What then of the questions I began to raise about liberal education back 
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at Chicago State? What do we mean, or rather, what should we mean when 
we advocate liberal education for today’s students, most of whom have come 
to college expecting that college will prepare them not to be scholars, but for 
opportunity and success in the world of work? 

I have already indicated that I believe liberal education begins by taking 
seriously the issues and questions in the students’ own minds. But what if the 
question they most want to pursue is how to do well in their jobs? Does that 
automatically put them beyond the true sphere of the liberal arts, except of 
course for the requisite dollop of general education courses necessary (on most 
campuses) to earn the degree? 

In 1976, I believed that a liberal education could — and should — be 
transformative because college had been so illuminating and life-enhanc- 
ing for me and most of my friends. Today, drawing on work being carried 
out across the United States, I am even more persuaded of the value and 
power of a liberal education, both for individual students and for a society 
that depends entirely on human capability for its economic future and its 
civic vitality. 

To sustain that conviction, however, I have had to break free of some of 
the most fundamental assumptions I had previously held about liberal educa- 
tion — especially the idea that it can (and should) occur exclusively through the 
study of arts and sciences disciplines. The twentieth-century ideas about liber- 
al education — which confined it to selected institutions, to selected disciplines, 
or to general education primarily — are no longer helpful to the larger cause of 
providing a life-enhancing education to all our students, especially all those 
“recently included” students who, only a few decades ago, would not have 
been in college at all. 

Yes, our students have largely come to college because they want to earn a 
good living. But these motivations are not an impenetrable obstacle to their 
liberal learning. Rather, their motivations should be seen as an opportunity for 
expanding their understanding of what they want to accomplish through their 
work and of the connections between their intended work and the larger soci- 
ety of which they are a part. 

My own education gave me, as I have tried to make clear, a set of tools 
that I could use to enlarge my own worldview and pursue my own significant 
interests. And that, I am persuaded, is what a liberal education needs to do for 
all our students, especially those who come to college convinced that the only 
point of their studies is to prepare for a job. 

A student who comes to college, for example, seeking to move from sales 
to marketing, or from bookkeeping to accounting, is no more beyond the pale 
than I was at eighteen, with my shallow conviction that the magic of miracles 
could “prove” fundamental points about divine intention and religious obliga- 
tion. Miss Brock made my initial questions a point of departure for further 
study and, by taking my actual questions seriously, she helped me to enlarge 
them. Something similar is possible for all our students, but only if we are will- 
ing to start with their actual mindset, which is so often very specific, very liter- 
al, and very practical. 
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One way to better appreciate our students’ job concerns, I suspect, is to 
think harder about the connection between our students’ obsession with “get- 
ting a job” and the journeys we ourselves have traversed toward professional 
competence. As a moment’s reflection makes clear, after all, the problem I 
myself was facing in my initial year at Chicago State was that I was complete- 
ly underprepared for the job I actually had. It is ironic that my Harvard cre- 
dential opened the door to an appointment at Chicago State even though 
Harvard had not provided me with even a moment’s rehearsal about how to 
connect my learning with my job. 

My graduate program preferred to prepare me for work I might do — 
path-breaking scholarship — rather than for the work I almost certainly would 
do — teaching novice students who, like most graduates of our nation’s public 
schools, need a huge amount of help to reap the full benefit of college. Nor 
was I alone. Even today, the most elite graduate programs remain largely 
indifferent to the actual work their students will do as faculty members, as 
campus leaders, or, increasingly often, in jobs beyond the academy. 4 

The questions I faced at Chicago State emerged not because I was espe- 
cially philosophical, and certainly not because I was interested in defining lib- 
eral education for its own sake. They emerged because I was negotiating daily 
between my students’ own priorities and the stated intention of our program 
to give them a liberal education. 

I went off on the intellectual journey I pursued because I was working, and 
because I wanted to do a good job — not only for my students but also for 
myself. My notions of the goal at hand were rooted in my own experiences of 
transformative learning, but it was extremely clear to me that, given my stu- 
dents’ very different experience of college (non-resident, working while learning, 
often much older, etc.), I would have to find different points of connection 
between their priorities and the realm of the liberal arts. 

Over time, I have come to believe that the root problem I confronted was 
the liberal arts’ self-imposed identity as “non- vocational.” The roots of this 
identity are worth pursuing; they go back to the twining of the liberal arts tra- 
dition with religious institutions, as well as to the embrace of ancient Greek 
philosophy. But that is a topic for a different essay. 

Here, I want to make the point that the positioning of liberal education as 
the opposite of vocational preparation is both willfully deceitful and foolishly 
self-defeating. It is deceitful because the actual outcomes of a liberal education 
(see fig. 2) are in fact a powerful form of preparation, not just for a particular 
job, but for the larger project of navigating a complex and fast-changing econ- 
omy in which the typical college graduate will hold many jobs, in different 
industries, under different employers. 

As my own saga shows, I had the skills I needed to “figure out” what I 
needed to do as a teacher and educator, and perhaps equally important, I had 
internalized a notion — from home and college — that if I was doing something 
I ought to do it as well as possible. It would have been better if I had had 
some preparation in connecting my schooling with my work, but even without 
that desirable apprenticeship, I at least had the intellectual skills and the sense 
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of personal and social responsibility to seek out help and eventually chart a 
productive course of action. I had acquired, in other words, the essential out- 
comes of a liberal education (see fig. 2). And these intellectual skills were, as 
they must be, initiatory rather than final; they created the context for continu- 
ing learning, much of which would be job-related. 

And, of course, my experience is replicated millions of times over in the 
work histories of all the graduates who also translate their arts and sciences 
studies into successful careers in every possible endeavor and every part of the 
globe. They may struggle initially to get onto a good career path. But once 
launched, they both contribute and succeed. 

So it is in fact misleading to assure college students that the liberal arts 
and sciences are “non-vocational.” But it is also profoundly irresponsible — a 
willful renunciation of our responsibility — not to think carefully about the 
kind of influence educators and scholars actually hold within the wider world. 

The fact of the matter is that most high school graduates will embark on 
postsecondary education, and most of those who enter college will work for all 
or a significant portion of their lives. We have an opportunity, therefore, to 
make the liberal arts and sciences a powerful resource, not just for an individ- 
ual, but for the world our graduates create through the work they do and the 
values they bring with them to that work. 

When we self-segregate the liberal arts and sciences from this wider 
world — insisting firmly on the value of learning for its own sake rather than 
for any vocational or instrumental purpose — we surrender the opportunity to 
examine with care how the values of the arts and sciences play out in the 
world of action, and how they might play out if we brought more mindfulness 
to these connections. 

After I left Chicago State, I went to the University of Chicago, where the 
dean of my division assured me cheerfully: “We take great pride here in teach- 
ing absolutely nothing useful.’' I felt a chill when he said this and I find that 
sentiment, which is held firmly by many humanists (and recently promulgated 
widely by Stanley Fish in The New York Times-*), chilling still. 

Students who major in arts and sciences fields deserve to believe that they 
are pursuing studies that are intensely useful. And they need opportunities to 
connect their learning with the world of action, whether through internships, 
field-based projects, service learning, or the jobs so many of them already 
hold. They should not have to wait until they are out of college and on the 
job to think through the connections between their chosen fields and the work 
they will actually do. The connections will be there. The campus should play a 
role in illuminating and enriching them. 

The majority of American college students do not, however, major in the 
arts and sciences at all. Like my students at Chicago State, they are choosing 
fields of study that are more transparently connected to the world of work. 

On campuses where this is the pattern, we have a double challenge. The first 
is to recognize that the outcomes of a liberal education (see fig. 2) both can 
and should be addressed in professional and career fields. Whatever the field, 
it requires its own version of the intellectual and practical skills that are neces- 
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sary elements in successful practice; it depends for its integrity on the sense of 
personal and social responsibility — the examined choices — its practitioners 
enact; and it faces its own “big questions” about its role in the wider world. 
Broad knowledge, advanced skills, ethical and civic responsibility, the ability to 
integrate one’s learning in new contexts: these are the hallmarks of a liberal 
education, and these are also the prerequisites for productive practice in every 
sphere of endeavor. 

The second challenge for campuses where arts and sciences majors are 
less commonly chosen is to tear down the silos that too often segregate the 
“true liberal arts” from professional and career fields. There are extraordinary 
opportunities both for scholarship and for creative teaching when faculty come 
together, across disciplinary boundaries, to link the insights of the arts and sci- 
ences with the challenges that face real-world practitioners in every sphere: 
health, business, government, public policy, technology, education. Big ques- 
tions and significant opportunities for shared work abound in every field of 
endeavor. And arts and sciences faculty can play a strategic role in helping 
practitioners see those questions in new contexts and with much deeper 
insight. Conversely, the liberal arts have much to gain from studying the work 
of faculty who take seriously their role in the formation of professionals, in 
preparing graduates to work and contribute, ethically and creatively, to their 
chosen fields and through their work, to their communities. 

Fulfilling the promise of a liberal education — for our time and our stu- 
dents — calls on us to approach our disciplines and our teaching in new ways 
and with a new concentration on the world of action. Our students need this 
from us. And so does our society. 


NOTES 

1 The vocabulary surrounding liberal education and liberal arts education is 
confused — and confusing. To cut through the thicket, the association I 
lead has developed a “Guide to Frequently Confused Terms.” See fig. 1. 

~ Readers can learn more about LEAP by visiting AAC&U’s website. The 
signature report from the LEAP initiative is College Learnmgfor the New Global 
Century , published by AAC&U in 2007, which outlines goals for a twenty- 
first-century college education, presents principles of excellence that build 
from best practice in higher education, and argues that the aims of liberal 
education should apply to all fields of college study, including professional 
and career fields. In 2008, the LEAP initiative released a study by George 
Kuh, High-Impact Practices: What They Are, Who Has Access to Them, and Why 
They Matter , which provides evidence that engaged or high-effort educa- 
tional practices — e.g., learning communities, undergraduate research, cap- 
stone experiences — have “compensatory” benefits for students who start 
farther behind academically and for students from groups with high college 
drop-out rates. The LEAP initiative provides numerous resources to cam- 
pus leaders and faculty working to create a more purposeful and empower- 
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ing educational experience for today’s students. 

^ Give Students a Compass is one of the LEAP “Principles of Excellence” and 
is also the title of a multi-state AAC&U LEAP project, supported by the 
Carnegie Corporation of New York, State Farm Companies Foundation, 
and the Lumina Foundation for Education, to map expected outcomes for 
liberal education across the college curriculum. 

^ From 1989 to 2003, AAC&U worked directly with graduate schools and 
some of their departments on new designs to prepare graduate students 
for their roles in teaching and academic leadership. The initial work was 
funded by a pilot grant from the Fund for the Improvement of 
Postsecondary Education (FIPSE). From 1993 to 2003, this work was 
expanded through major grants from the Pew Charitable Trusts, the 
National Science Foundation, and Atlantic Philanthropic Services. Over 
time, AAC&U and its partner organization, the Council of Graduate 
Schools, worked directly with more than forty graduate universities to 
broaden their approach to graduate student preparation. While this work, 
and comparable efforts by the Carnegie Foundation for the Advancement 
of Teaching, was valuable to those influenced by it, the gap between grad- 
uate education and the needs of today’s undergraduates remains wide 
indeed. Moreover, the average new PhD is as ignorant today as I was in 
1976 about the traditions of liberal education and their meaning in society. 

^ See, for example, “Will the Humanities Save Us?” where Fish writes that 
“to the question ‘of what use are the humanities?’, the only honest answer 
is none whatsoever. And it is an answer that brings honor to its subject.” 
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MAKING THE CASE FOR DISCIPLINE-BASED 
ASSESSMENT 

RACHELLE L. BROOKS 


COLLEGE AND UNIVERSITY FACULTY have been increasingly recog- 
nizing the value of some sort of evaluation and assessment of the impact of 
teaching strategies. Perhaps initially encouraged by administrators, accredita- 
tion organizations, or disciplinary associations, once faculty members are 
introduced to assessment many are willing to engage with it to see if it can be 
used for improving their classroom practices or teaching techniques. There is 
often, however, a wide gap between the conceptual agreement that assess- 
ment has some value and the devising and employment of a concrete assess- 
ment strategy. 

How can wc best gather the information we would like to have for learn- 
ing about the strengths and weaknesses of our teaching? How can we know 
whether our students are really learning what we think and hope they are 
learning? What strategies should we use? What methods? In many cases, the 
answers to these questions can emerge from one’s own disciplinary framework. 
By working with others in one’s discipline and by focusing on the learning 
goals and methods central to it, faculty can devise an assessment program for 
improving teaching and learning in their classrooms. Such efforts often also 
fulfill larger institutional desires to demonstrate student learning outcomes. 

In the following pages I argue that this sort of assessment within the disci- 
pline, rather than that which attempts to exist outside or above all disciplines, 
has the potential to be more “actionable” for faculty and other institutional 
policymakers (that is, they are able to identify actions to take in their class- 
rooms or programs based on the results of the assessments). Discipline-based 
assessment is also more sensitive to variations in learning between groups of 
students . 1 Assessments that do not attend to disciplinary differences or that are 
not consciously rooted in specific disciplines have inherent weaknesses that 
limit their validity and their utility for the improvement of teaching. Unfortu- 
nately, most large-scale, institution-wide assessment efforts cannot be used at 
the disciplinary level because they limit their study to samples of students with 
too few individuals to capture learning in a specific major. They attempt to 
draw conclusions about all students, irrespective of their majors. In addition, 
such assessments often possess unexamined disciplinary biases that create an 
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uneven playing field for making meaningful comparisons between majors. 
Questions in an assessment have disciplinary “leanings” that are ignored at 
our peril when setting out to measure student cognitive growth. 

In this essay I argue that, while remaining rooted within the existing 
framework of one’s discipline, assessments can be devised for both content- 
based knowledge (which is routinely evaluated by faculty) and the broader, 
more generalized learning outcomes often associated with a liberal arts educa- 
tion, such as critical thinking. In creating such an assessment it is possible to 
avoid getting caught up in either the innumerable specificities of each stu- 
dent’s college experience or overly broad generalities about all college stu- 
dents. By way of example I discuss a study which involves working with facul- 
ty to engage in this type of discipline-based outcomes assessment to learn 
about the broader learning outcomes of their disciplines. This example of 
assessments for classics and political science helps to illustrate the distinction I 
make between assessment in the discipline and assessment of the discipline. 

Student Learning Happens in Disciplines 

Higher education in the United States is part of a complex ecology of 
institutions, policies, programs of study, and student experiences.- Any 
attempt to measure undergraduate collegiate learning must make some effort 
to attend to this diversity of educational options. Jeffrey Smith’s articulation of 
some of the types of heterogeneity in higher education is instructive: “... we 
can think about students having a variety of possible outcomes that depend on 
their own characteristics (measured and unmeasured), the particular college 
they attend and its characteristics (and the characteristics of its student body 
as a whole), and the particular program or programs they undertake” (133). 
Thus, the choices students make about the institution at which they will study, 
the learning opportunities they will pursue there, and the disciplinary major 
with which they will engage all bear on the educational outcomes they will 
have attained upon completion of their degrees. It is not the case that all stu- 
dents at a single institution will experience similar levels of cognitive change 
during college, no more than it is the case that they will have identical experi- 
ences while enrolled. 

George Kuh, a leading scholar of undergraduate education, has identified 
a set of “high-impact educational practices” that research has shown improves 
student retention rates and increases students’ engagement with their colle- 
giate education. These practices include first-year seminars, common intellec- 
tual experiences (such as a “core” curriculum), learning communities, writing- 
intensive courses, collaborative assignments and projects, undergraduate research, 
diversity/ global learning, service learning, internships, and capstone courses or 
projects. There is wide divergence in the extent to which these practices are 
employed across the spectrum of undergraduate institutions. And even once in 
place, unless they are required, not all students will choose to endeavor at 
these types of learning experiences, nor will they be available at similar levels 
or be of comparable quality across all majors. 

It is notable that several of these “high-impact” practices are most 
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likely to take place at the discipline-level, within one’s major. Undergrad- 
uate research projects, collaborative assignments, writing-intensive 
courses, 3 and capstone projects are all educational activities directed by 
individual faculty members. The benefits to undergraduates of faculty- 
driven, discipline-based work was also noted by Richard Light in his 
decade-long study of college seniors. He writes, “... students who get the 
most out of college, who grow the most academically, and who are happi- 
est organize their time to include activities with faculty members, or with 
several other students, focused around accomplishing substantive academic 
work” (10). Such work will surely vary depending on the faculty member’s 
discipline. So, while a physics major may not have access to many writing- 
intensive courses in her discipline, she may have numerous opportunities to 
engage in research projects with faculty. Such differences by field of study 
impact students’ learning outcomes in measurable ways, with one student 
growing at ease with composing an essay or using writing to summarize a 
body of work, while the other emerges confident in her ability to identify 
and evaluate multiple possible solutions to unfamiliar problems encoun- 
tered in an experiment. 

Why “One-size-fits-all” Assessment Really Doesn’t 

The “one-size-fits-all” approach to undergraduate assessment, where all 
students are assessed with the same instrument and no distinctions by disci- 
pline are made, has found a niche at many institutions. It gained a great deal 
of momentum when in 2006 the Secretary of Education’s Commission on the 
Future of Higher Education called on higher education to demonstrate and 
report on learning outcomes. In its report, entitled A Test of Leadership: Charting 
the Future of U.S. Higher Education , the Commission urged institutions to employ 
assessments of student learning that would enable comparisons between insti- 
tutions. However, the ability to compare at the college or university level 
comes at a serious cost to the local utility of the assessment (Banta, “A 
Warning”). It usually means using a standardized test to generalize about all 
undergraduates as a group at an institution, regardless of their major or any 
other educational choices, including their participation in any of the afore- 
mentioned “high-impact” activities. Findings from an institution-wide assess- 
ment are also often impossible to relate back to specific instructional tech- 
niques and other types of pedagogical or curricular choices that faculty have 
the power to control. Such assessments offer faculty members little, if any, 
information about how they might improve students’ educational experiences 
or about what they may be doing particularly well in terms of their instruc- 
tional practices. 

Assessments for institutional comparisons also require measuring learning 
in a uniform way for institutions. The choice of a particular, standardized way 
to measure learning is also a choice about the types of learning that matter, 
which has disciplinary implications. As noted earlier, there is an inherent, 
often unrecognized disciplinary bias in any measurement of student learning. 
This is perhaps easiest to see when examining students’ competencies with 
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direct measures of learning. A test of general written communication abilities 
or mathematical competencies privileges students in particular majors in obvi- 
ous ways. For example, those in science, technology, engineering, or math will 
likely score very well on an assessment of mathematical ability. Alternatively, 
assessments of more general learning outcomes such as critical thinking tend 
to use items tied to the “general education” curriculum in an effort to ensure 
their relevance to a wide population. The underlying assumption appears to 
be that these broad outcomes such as critical thinking should be assessed with 
broad, widely applicable measures. Thinking and reasoning skills, however, are 
developed through numerous educational experiences, many of which are dis- 
cipline-specific (McPeck; Ennis). Students in humanities, science, and social 
science disciplines may all be trained to evaluate evidence, draw conclusions, 
and make arguments, but some are taught to do so using texts while others 
become proficient at leveraging quantitative data for their analyses. To ignore 
disciplines when assessing thinking and reasoning skills is an attempt to 
divorce them from their developmental roots. Further, it is hard to imagine a 
type of assessment that is truly content-neutral. For example, assessments of 
students’ thinking abilities that ask them to answer questions in essay form are 
biased in favor of those who are better trained in writing or for whom English 
is a first language. Trudy Banta, who has spent nearly three decades examin- 
ing student learning outcomes and various ways to measure them, has repeat- 
edly argued that “[n]o standardized exam is truly content free, and if it were, 
it would be a better test of general intelligence than of what is learned in col- 
lege” (“Reliving the History” 4). 

One strategy for avoiding standardized tests of collegiate learning is to use 
a survey asking students about their learning in college. These surveys gauge 
students’ perceptions of their improvement in certain areas and their satisfac- 
tion with their college experiences. Unfortunately, there is also evidence that 
disciplinary biases are present even when these indirect measures of learning 
are employed, such as in student opinion surveys about educational gains and 
engagement in college. 

A recent study of survey results from approximately 58,000 students in the 
University of California (UC) system provided an opportunity to examine the 
extent to which majors are related to how students perceive their educational 
outcomes and their level of educational engagement (Chatman). Because the 
data were obtained from students at multiple campuses in the UC system, it 
was possible to separately examine the influences of campus and major. The 
principle finding of the study was that the major had a more important 
impact on student perspectives and engagement than did the campus at which 
students were enrolled. The author of the study, Steve Chatman, notes that 
“[t]he most important result is that academic experience and student engage- 
ment varies by program of study in predictable ways” (1 1). For example, he 
found that students in the humanities and social sciences reported higher levels 
of satisfaction with their educational experience than did those in mathemat- 
ics and computer science or engineering. He asserts that “differences [by aca- 
demic discipline] would lead to misleading conclusions when comparing one 
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program to a campus average and when comparing one campus to another” 
(15). Ultimately, any campus- or institution-wide averages will be dictated by 
the proportion of students in particular majors. In this case, campuses with 
more math and engineering majors would have lower satisfaction scores. 
Chatman writes: 

Modern public research universities are academically diverse and, 
by publicly supported agreement, serve extremely diverse popula- 
tions. The accountability strategies that have been at least partial- 
ly successful in improving elementary and secondary education 
cannot be easily generalized to postsecondary study because post- 
secondary education is more complex by at least an order of 
magnitude. Elementary schools offer few course choices, second- 
ary schools several more within a few program tracks, and post- 
secondary institutions offer a hundred or more academic majors 

and thousands of courses If public accountability demands 

comparative performance [between institutions], then the unit of 
analysis for performance should be the academic discipline. (3) 

Assessment in, but not of, the Discipline 

Given the disciplinary biases present in both direct and indirect measures 
of student learning, it would seem that we are far better off recognizing and 
negotiating disciplinary distinctions when approaching questions about the 
cognitive value of a college education. Rather than ignoring the “confound- 
ing” variable of the disciplinary major when examining student learning, it is 
possible to explicitly address its role in the development of cognitive growth. 
Such an approach requires asking hard questions about the learning goals for 
a major, what a major offers students, how a set of required and elective 
courses contribute to the overall goals, and how best to uncover the potential 
contributions of a major to a student’s overall learning. But this approach is 
worth the effort because assessment that fails to do so risks providing little use- 
ful information for faculty at best, and of generalizing false conclusions about 
learning outcomes at worst. 

While many discussions within professional associations can be con- 
tentious when trying to identify a concrete set of “learning objectives” for 
undergraduates majoring in a particular discipline, smaller groups of faculty, 
such as chairs of those teaching in liberal arts colleges, may be better able to 
engage in such discussions by asking, “What is it that we do best?” or “What 
do we want all of our majors to be able to do well?” or “What have we seen 
over our careers that makes our students who major in our disciplines stand 
out from other students?” Answers to such questions can begin to shed light on 
how a major might contribute to broader learning outcomes, which arc devel- 
oped in a discipline, but are not unique to it. 4 These outcomes are learning 
that stays with a student long after specific course content is forgotten, and the 
type of learning treasured in the liberal arts because it is widely applicable to 
many different occupational pursuits. Michael Roth, president of Wesleyan 
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University, articulates specifics for this type of learning in the humanities, say- 
ing that an education in the humanities allows one to develop skills in 

reading, writing, reflection, and interpretation that are highly prized [We 

also] teach a set of skills, or an attitude, in the humanities . . . the set of skills 
that usually goes under the rubric of critical thinking.” 

To assess the extent to which students are developing these skills within a 
particular major requires honing in on which outcomes are most applicable to 
the discipline, and then identifying an appropriate way to observe them in stu- 
dents. Once one sets out to determine whether a type of learning has taken 
place, how students are asked to demonstrate that learning matters a great 
deal, and I contend that the method chosen should take into account discipli- 
nary training. For example, it may be the case that students in the humanities 
have processes for thinking critically that best lend themselves to working on 
problems in those disciplines. It is also very likely that students will draw on 
their specific curricular training to demonstrate their cognitive abilities when 
asked to do so. It would follow that attempts to evaluate critical thinking out- 
comes for these students with assessments lacking humanities content would 
result in a good deal of measurement error. An effort to assess the critical 
thinking ability of humanities students that does not include an opportunity 
for them to draw on the reading, writing, reflection, and interpretation skills 
they cultivate in their majors will not provide the best forum to deeply engage 
their thinking and reasoning abilities. The resulting observation and measure- 
ment of these abilities would be weakened. 

While there is no shortage of assessment instruments readily available that 
target specific and general learning outcomes, none uses the discipline as the 
starting point for testing broad learning goals of college.'* Rather, the assess- 
ment tools fall into one of two categories. When general learning outcomes 
are the target of the assessment, student disciplinary concentrations are 
ignored, and the tool is considered “content- neutral.” Questions in the assess- 
ment try to uniformly measure the overall contribution of four or more years 
of college to students’ cognitive or skill-based abilities. Thus, the assessments 
to test reasoning and thinking with essay questions about a contemporary 
social issue, for example, or standardized multiple choice questions. When 
assessments are designed to address disciplines, they target content-specific dis- 
ciplinary knowledge, not general learning outcomes such as reasoning abilities. 
These standardized, subject-specific tests measure what students have learned 
about their discipline through courses in their major. So, definitions of key dis- 
ciplinary terms or scientific principles are included as questions in such assess- 
ments. Yet a third way of thinking about assessment is possible — one that 
draws on disciplinary approaches and methods of learning while targeting 
general learning outcomes. This type of assessment, which is rooted in a disci- 
pline, but is not designed to be an assessment of disciplinary content, has the 
potential to avoid many of the limitations described above, and can be 
employed by faculty with an interest in better understanding the learning that 
takes place in their courses and departments. 
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An Example: Assessing Thinking and Reasoning for Classics and 
Political Science 

I began a project in 2007 to assess general learning outcomes in two disci- 
plines, political science and classics. This multi-institutional study seeks to 
examine and document the development of general learning outcomes using 
tools that draw from disciplinary learning but do not measure disciplinary 
content. Because the discipline is central to the focus of this research, faculty 
involvement has been key to: 1) identifying appropriate outcomes for investiga- 
tion; 2) devising the instrumentation; 3) creating scoring rubrics for the assess- 
ment; 4) supervising the grading of components of the assessment; 5) identify- 
ing appropriate samples of majors within each participating institution; and 6) 
recruiting students to participate in the study. 

To begin this work, faculty from all participating institutions formed a col- 
laborative, multi-institutional, cross-disciplinary consortium that included me 
and my research team. Using the discipline of classics as a case study, the 
study started with a small group of faculty from five institutions who were 
asked to identify broad cognitive outcomes cultivated in training within clas- 
sics. We then zeroed in on a few specific outcomes and began developing 
instrumentation for their measurement. In its focus on broader cognitive abili- 
ties, such as critical thinking, the project strove to avoid testing for content- 
based, domain-specific knowledge. 

Ultimately faculty in both classics and political science developed essay 
questions to test for critical thinking abilities. Essay questions were considered 
appropriate means by which students in these disciplines could demonstrate 
their critical thinking abilities, since the reading and writing of essays, books, 
and articles are principle ways in which knowledge is gained and communicat- 
ed in these disciplines. The questions drew on content from their disciplines 
but did not require extensive discipline-based knowledge to answer. For exam- 
ple, classics students were asked to consider pictures of artifacts from an 
archaeological dig as a basis for drawing conclusions about an unknown civi- 
lization. Political science students were asked to make inferences about the 
political and governmental conditions in a fictional country that were support- 
ed by newspaper and magazine clips they received. 

The essay questions were pre-tested on a group of majors and non-majors 
and revised for clarity and precision. The resulting assessment was adminis- 
tered to approximately 1,000 first- and second-year college students at a dozen 
institutions. We will be reassessing these students in their fourth year to 
observe the amount of change in thinking and reasoning that takes place in 
students of different majors, with special attention to classics and political sci- 
ence. Scoring rubrics were also devised by the faculty to grade the essays, and 
were tested and revised to ensure they could measure general critical thinking 
abilities and not disciplinary knowledge. Ultimately, scores using these rubrics 
designed for the specific essay questions correlated well with a “generic” critical 
thinking rubric that could be used to score any essay (Facione and Facione 1994). 6 

Despite frequent references in the higher education literature to faculty dis- 
engagment with assessment work (Palomba and Banta, Assessment Essentials ; Bok), 
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faculty participation in this project continues to be both strong and easy to 
obtain. The perceived authenticity of the assessment research likely contributed to 
faculty enthusiasm. That is, it is work that is conceived through collaboration, 
grounded within a discipline, administered locally rather than as part of a large 
national study, and reported on with the objective of aiding faculty in better 
understanding the impact of their pedagogical and curricular choices. ^ 

Its scholarly significance lies not just in the specific findings of under- 
graduate cognitive ability available from the assessment, but also in the 
methodological approach to the measurement of broad cognitive outcomes 
through disciplinary content, and in the successful operationalization of that 
method with deep and broad involvement of faculty with no prior experi- 
ence with this form of assessment. Assessment studies that offer an authentic 
experience for faculty members can be highly successful, despite the chal- 
lenges of conducting assessment in colleges and universities, which must 
operate within the complex social and cultural ecologies of higher educa- 
tion. This study explicitly addresses the role of the disciplinary major — too 
often seen as a “confounding” variable in an assessment of broad cognitive 
outcomes — in the development of critical thinking, and does so with a 
method that calls upon faculty in those disciplines to think carefully and 
explicitly about the contribution of their disciplines to gencralizable, lifelong 
learning outcomes. 

Conclusion: Faculty Participation is Essential 

Assessment projects of this sort have much to offer faculty who are ready 
to investigate the teaching and learning taking place in their disciplines. 
Likewise, as stewards of their disciplines, all faculty members have something 
to offer assessment projects when questions of their students’ learning out- 
comes are at hand. This work comes down to questions about how best to 
educate students in college, which ultimately must be answered by faculty. 
Derek Bok reminds us that “... faculties and their academic leaders have con- 
siderable freedom to shape undergraduate education and to teach their stu- 
dents as they choose. The manner in which they exercise their discretion and 
the values and attitudes they bring to the task have much to do with the quali- 
ty of education students receive” (34). 

Unfortunately, as Susan Engel observes in her discussion about the role of 
teaching in college students’ success, there are few support structures upon 
which faculty can anchor efforts to assess their teaching and their students’ 
learning. She writes: 

Good teaching is essential to good colleges. And while our colleges 
and universities vary greatly in how much they value teaching, 
even the ones who care about it most do little to make sure it hap- 
pens. Individual teachers must articulate, and constantly reconsid- 
er and reconfirm their educational goals, using those goals as a 
guide to evaluating their teaching, day in and day out. (194) 
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But unlike much of the work done by faculty in academia, this need not 
be a purely solitary effort. When disciplines come to the forefront, colleagues 
can be found both inside and outside one’s institution. Conversations turn to 
questions of what matters most to a discipline, and such discussions can be 
among the most rewarding for those who have dedicated their careers to fur- 
thering its intellectual goals. 

Thank you to William Hayward for his thoughtful comments on an earlier draft. 


NOTES 

1 Note that discipline-based assessment is much less relevant for those insti- 
tutions and individuals whose primary concern is the early learning that 
takes place in students’ first or second college year, when most have not 
yet declared a major. Also, it is irrelevant for two-year colleges and those 
institutions with low graduation or retention rates. It is most applicable to 
those interested in overall collegiate outcomes, sometimes known as the 
“value added” of a college education. 

^ Light, Cox, and Calkins extend the examination of higher education’s 
context by examining the numerous pressures the larger society exerts on 
the professoriate in the US, such as globalization and the shift to a knowl- 
edge-based economy. They accurately note that these, too, have an impact 
on teaching, learning, and assessment. 

*’ I refer here not to courses designed to fulfill a college’s writing require- 
ment, usually taken in the first year of college. I mean instead those 
courses found most often in humanities and social science disciplines 
which almost exclusively utilize essay exams and/ or term papers to estab- 
lish a student’s grade. 

^ This is not meant to discount the important efforts faculty make to assess 
the content learning that takes place in disciplines. Ideally, both general 
learning outcomes and domain-specific outcomes can be integrated into 
faculty-driven assessment approaches. 

^ See the National Center for Postsecondary Improvement’s “Inventory of 
Higher Education Assessment Instruments” webpage, Borden and 
Kernal’s Measuring Quality in Higher Education website, and Schechter’s web- 
page on “Internet Resources for Higher Education Outcomes 
Assessment” for extensive listings of assessment instruments and method- 
ological approaches. 

® The generic rubric we used was Peter A. Facione and Noreen C. Facione’s 
“Holistic Critical Thinking Scoring Rubric.” 

^ See also Palomba and Banta, Assessing Student Competence in Accredited 
Disciplines. 
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WHERE HAS ASSESSMENT BEEN IN THE MODERN 
LANGUAGE ASSOCIATION? A DISCIPLINARY 
PERSPECTIVE 

ROSEMARY G. FEAL, DAVID LAURENCE, AND STEPHEN OLSEN 


MANY PEOPLE IN THE ACADEMIC community expressed surprise when, 
in 2008, Modern Language Association (MLA) President Gerald Graff wrote 
an MLA Newsletter column that spoke out in favor of assessment. Wasn’t assess- 
ment the purview of other disciplinary associations, such as the National 
Council of Teachers of English (NCTE) and the American Council on the 
Teaching of Foreign Languages (ACTFL), which have had a deep stake in fos- 
tering discussions of student learning (how to define, promote, and measure 
it)? It seems logical that associations with missions centered on the teaching of 
writing or of language acquisition would devote significant resources to the 
issue of assessment. Although many members of the MLA offer courses that 
might not be classified as “writing” or “language” courses per se, these faculty 
members typically teach aspects of writing and language and are used to 
thinking about how to document student progress. In this brief piece, we want 
to demonstrate that the MLA has in fact been deeply concerned with matters 
of assessing student learning and has produced a very useful body of work to 
support that concern. 

The MLA’s work on assessment has been undertaken primarily by the 
Association of Departments of English (ADE) and the Association of 
Departments of Foreign Languages (ADFL), organizations that have been 
parts of the MLA since the 1 960s and whose chief purpose is the exchange of 
information among departmental administrators. With upwards of 1,500 
member departments, ADE and ADFL function as central sources of support 
for educational programs in English and other modern languages at colleges 
and universities throughout the United States and Canada. They offer depart- 
ment chairs a community of peers and access to the extensive information 
resources of the MLA, while providing both departments and the MLA itself 
invaluable perspectives on how intellectual, curricular, and public policy devel- 
opments are affecting local programs and the profession nationally. Since the 
mid-1980s at least, assessment of student learning has been a regular topic for 
consideration at the summer seminars for department administrators that 
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ADE and ADFL sponsor each June and the subject of dozens of articles in 
th e ADE Bulletin and ADFL Bulletin, the associations’ journals. 

Some background is in order before we discuss the MLA’s part in assess- 
ment in the fields of English and other languages. 1 Assessment of student 
learning — and education more broadly — came center stage as a public policy 
issue of national concern following the release in 1983 of A Nation at Risk, the 
report on the state of American education from then-Secretary of Education 
Terrell Bell’s National Commission on Excellence in Education. Not coinci- 
dentally, A Nation at Risk circulated in a political context of increasing stress on 
state and local budgets — stress created in no small part by unfunded mandates 
that federal legislation imposed on state and local governments and that, as 
one consequence, cast education, corrections, and Medicaid as competitors for 
increasingly tightened state and local tax resources. Higher education is gener- 
ally the largest single pot of discretionary money in a state budget, while cor- 
rections and Medicaid are mainly non-discrctionary expenditures. As correc- 
tions and Medicaid started to claim notably large and alarmingly enlarging 
shares of state budgets, legislators began to question just what public expendi- 
tures on the education portion were purchasing — and to look to assessment as 
a tool for finding out. 

For higher education, the issue emerged most consequentially in the con- 
gressional debate surrounding the reauthorization of the Higher Education 
Act in 1992, especially Title IV, the part of the legislation dealing with feder- 
al student grant and loan programs. The debate that took place during the 
reauthorization episode of the early 1990s established terms for a public poli- 
cy discussion that have persisted for two decades and that will be entirely 
familiar to anyone who is following the arguments today about higher educa- 
tion and the public or publicly subsidized or guaranteed dollars colleges and 
universities receive in the form of tuition through their participation in Title 
IV student-aid programs — approximately #113 billion in 2009 (United States, 
“Federal Student Aid,” 5). Then, as now, congressional attention focused pri- 
marily on for-profit institutions but tended to implicate the entire system, not- 
for-profit liberal arts and science institutions as well as for-profit vocational 
schools and institutes. Then, as now, questions arose about the default rates on 
student loans, many of which are backed by government guarantees. Then, as 
now, a basic impetus was to create legislative provisions that would oblige insti- 
tutions receiving student-loan tuition dollars — a debt burden that often puts 
the student financially at risk — to document that the educational programs 
students undertook were actually delivering the education advertised. 

The Higher Education Act made institutional accreditation the criterion 
for institutions’ eligibility to participate in Title IV student-aid programs. In 
the 1 992 debate about reauthorization, Congress came close to decoupling eli- 
gibility to participate in Title IV programs from accreditation and creating a 
new and unprecedented system of direct governmental supervision. By agree- 
ing to implement assessment and documentation of student learning outcomes 
as part of the accreditation process, higher education’s voluntary accreditation 
system succeeded in preserving the tie between accreditation and eligibility for 
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institutions to participate in Title IV student-aid programs. Ever since, assess- 
ment and requests for departmental plans and procedures for documenting 
student learning outcomes have been visible accompaniments of institutional 
accreditation reviews. 

In the aftermath of the 1 992 reauthorization debate and the new commit- 
ment to assessment of student learning exacted of higher education’s regional 
accreditation bodies, the ADE appointed an ad hoc committee “to investigate 
the issues and procedures of assessment in English and to consider what 
advice the ADE can usefully offer to departments and chairs engaged with the 
problem of developing assessment initiatives, especially initiatives focused on 
the documentation of student learning” (“Report of the ADE Ad Hoc 
Committee on Assessment” 1). In 1994 the ad hoc committee surveyed ADE 
member departments about their assessment practices and experiences, and in 
1 996 submitted its final report. The “Report of the ADE Ad Hoc Committee 
on Assessment” remains a vital resource for departments undertaking assess- 
ment initiatives and offers explicit advice about fostering a culture of evidence 
for documenting student learning in humanities departments. It provides a 
political and historical context for the demands being made on English and 
other humanities departments, reports on the experiences and initiatives of a 
variety of departments across the country, offers detailed guidance in selecting 
and presenting evidence, and presents a selection of “Hints, Tips, Words to 
the Wise, and Other Cautionary Observations.” The report advises faculty to 
think globally and act locally when responding to assessment mandates: 

Assessment indeed takes place in an environment of national eco- 
nomic trends and professional goal setting, but it also inevitably 
invokes a host of intensely individual institutional concerns — 
refining relations to original ethnic or religious institutional affilia- 
tions, raising issues involving the specific urban or regional popu- 
lations the institution serves, or prompting attention to unique 
excellences often fostered over several generations. The most con- 
vincing assessment will be generated where these well understood 
institutional particularities ground the discussion of economic 
realities and professional aspirations. (6) 

Attention to outcomes assessment also informs other ADE reports, such as 
the 2003 “Report of the ADE Ad Hoc Committee on the English Major,” 
which further articulates the diversity of English as a discipline and an admin- 
istrative unit. 

A key strength of English as a centerpiece of liberal education 
has surely been its breadth and adaptability. English departments 
have been a traditional locus of curricular experimentation, nur- 
turing and then sending out into the world such former fledglings 
as American studies, African American studies, and women’s 
studies. As John Gerber, a longtime chair at Iowa, wisely observed 
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more than twenty-five years ago: “English is not a neat, discrete 
discipline, but a congeries of subject matters that varies from 
place to place and time to time.” (72) 

The persistently polymorphous character of curricula and research in lan- 
guages, literatures, and cultures has implications for how faculty members in 
English and other language departments think about student learning and are 
inclined to assess it: 

It has never been assumed that studying English [for example] 
implies a specific course of study and a prescribed set of out- 
comes, whether outcome is defined as a standard body of knowl- 
edge or a standard display of skills. Local departments and facul- 
ty members have thrived in a context of broad freedom to deter- 
mine curricula and course design in the light of local knowledge 
informed by participation in the wider scholarly community, 
whose activity continually shapes and reshapes the arguments 
directing development of the discipline at large. One expects a 
major in English to acquire and refine skills of analytic reading, 
writing, and communication, but these skills are largely a by- 
product of the formal study of language and literature. . . . 

In English, certain obvious elements (spelling and punctua- 
tion, knowledge of grammatical terminology, names and dates of 
authors and their works) are easily incorporated as features of an 
outcomes system. But little more than functional literacy can be 
tested in this way — and we insist that baccalaureate education 
amount to a great deal more. Critical judgment, intellectual 
inventiveness, perspicacity of observation, depth of research, the 
capacity to draw on a broad range of relevant materials, analytic 
shrewdness, and originality in making sound connections and dis- 
tinctions — all are far less amenable to standardized measures. (80) 

The 2006 report of the MLA’s Ad Hoc Committee on Foreign 
Languages, “Foreign Languages and Higher Education: New Structures for 
a Changed World,” demonstrates how sustained reflection about curricu- 
lum and learning is concomitant with the useful consideration of student 
outcomes. The report articulates a new vision of foreign language educa- 
tion whose goal is translingual and transcultural competence achieved 
through a broad and more coherent curriculum “in which language, cul- 
ture, and literature are taught as a continuous whole” (3). Among the 
report’s recommendations are that departments “set clear standards of 
achievement for undergraduate majors in speaking, reading, writing, and 
comprehension and to develop the programming necessary to meet these 
standards” and “establish language requirements (or levels of competence) 
for undergraduate students majoring in fields such as international studies, 
history, anthropology, music, art history, philosophy, psychology, sociology, 
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and linguistics, as well as for students preparing for careers in law, medi- 
cine, and engineering” (7-8). 

The report’s articulation of what such standards and competencies might 
look like in “educated speakers who have deep translingual and transcultural 
competence” (3) can serve as an example of the complexity and subtlety of 
actual outcomes in humanistic inquiry: 

One possible model defines transcultural understanding as the 
ability to comprehend and analyze the cultural narratives that 
appear in every kind of expressive form — from essays, fiction, 
poetry, drama, journalism, humor, advertising, political rhetoric, 
and legal documents to performance, visual forms, and music. 
According to this model, which we present only as an example, to 
read a cultural narrative a student should: 

• Achieve enough proficiency in the language to converse with 
educated native speakers on a level that allows both linguistic 
exchanges and metalinguistic exchanges (that is, discussion 
about the language itself). 

• Have a solid command as well as an analytic knowledge of 
specific metaphors and key terms that inform culture. 

• Understand how a particular background reality is re-established 
on a daily basis through cultural subsystems such as: 

- the mass media 

- literary and artistic works as projection and investigation 
of a nation’s self-understanding 

- the social and historical narratives in literary texts, artis- 
tic works, the legal system, the political system, the edu- 
cational system, the economic system, and the social wel- 
fare system 

- local instances of major scientific and scholarly para- 
digms 

- sports or other leisure activities, the cultural metaphors 
these have created, and their relation to the national 
imagination 

- stereotypes, of both self and others, as they arc devel- 
oped and negotiated through texts 

- symbols or sites of memory in the broadest sense, includ- 
ing buildings, historical figures, popular heroes, monu- 
ments, currency, culture-specific products, literary and 
artistic canons, landscapes, fashion, and cuisine 

- major competing traditions such as views of the nation 
that are secularist or fundamentalist or religious 

- local historiography. (4-5) 
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The integration of assessment into the regular cycle of institutional 
accreditation has led to reflection on good practice for departments and the 
field across a range of viewpoints from the decidedly skeptical to the warmly 
evangelical. The Winter 2008 ADE Bulletin includes a special section on 
“Assessment Pro and Con,” featuring papers delivered at the 2007 ADE 
Summer Seminar for Department Chairs. The Fall 2008 ADEL Bulletin 
includes a cluster of articles arguing that the goals of the humanities can be 
supported by effectively designed assessments of student learning. As of this 
writing, a search on the category “assessment of student learning” returns a 
list of 135 articles in the ADE Bulletin archive; in the ADEL Bulletin archive a 
search on the term “assessment” returns a list of 54 articles. It is fair to say 
that faculty members in language and literature have engaged the formal 
assessment of educational programs through the documentation of student 
learning outcomes — considered as a systematic departmental practice — with a 
certain degree of wariness. But engage they have and engaged they are. In 
preparation for a panel on assessment of student learning in English at the 
2010 Summer Seminar East, the ADE surveyed chairs of its member depart- 
ments about their assessment projects and plans. Not surprisingly, 86% said 
that their department had implemented an assessment process, and 90% said 
assessment has the potential to improve student learning in their department’s 
programs (“ADE Survey”). 

In 2006, the MLA along with other disciplinary groups responded to an 
invitation from the Teagle Foundation to think about “the relationship 
between the goals and objectives of undergraduate concentrations in their dis- 
ciplines and those of liberal education” (“Report to the Teagle Foundation” 1). 
The report to the Teagle Foundation from the MLA’s working group recom- 
mended “an approach to structuring baccalaureate programs in English and 
other languages that combines four constitutional elements” — and goes on to 
list “a coherent program of study,” “teamwork among instructional staff 
members,” “interdepartmental cooperative teaching,” and “empirical research 
to assess the successes and shortcomings of the program” (3). These aspira- 
tions toward educational activities that are grounded in programmatic coher- 
ence and educational acquisitions that can be empirically documented are no 
less characteristic of faculties in language and literature; nonetheless, they are 
perhaps less frequently given public expression than are affirmations of a force 
in language and literature that maintains uneasy relations with whatever in 
education can be specified as outcomes. In a famous sentence at the conclu- 
sion of “Russian Writers, Censors, and Readers,” the opening essay in Lectures 
on Russian Literature , Vladimir Nabokov declares, “Readers are born free and 
ought to remain free” (12). Nabokov understood the essence of a reader’s free- 
dom to consist in the principle that “no director of conscience and no book 
club” — and, we might add, no scheme of rubrics and outcomes decided by a 
committee of experts — “can manage [a reader’s] soul” (1 1). If some MLA 
members maintain a certain characteristic wariness toward outcomes assess- 
ment, the wariness may only reflect their deep instinct to preserve and defend 
the reader’s freedom. That is, many of us cling to an awareness that litera- 
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ture’s educative force lies in some measure in its power to teach what freedom 
is precisely by propelling us into experiences of the reader’s freedom. 
Assessment presents the challenge of meshing or reconciling this instinctive 
bias for the reader’s freedom with the schematic nature of rubrics and out- 
comes that, defined in advance, would seem of necessity to circumscribe read- 
ing and compromise the reader’s freedom. But this is perhaps to say no more 
than that language and literature, and the study and teaching and learning of 
language and literature, stand in uneasy relationship not just to assessment but 
to the institution of organized education. An uneasy relationship need not be 
an unproductive one. Literature and the study, teaching, and learning of lan- 
guage and literature have created an educationally productive arena in the 
school over a long and distinguished history. The encounter of language and 
literature with assessment, that relative newcomer on the institutional scene, is 
doing the same. 


NOTES 

1 Sources for the history summarized in this paragraph and the two para- 
graphs following may be found in two ADE Bulletin “From the Editor” 
columns (Laurence [1994] and Laurence [1995]). See also Glidden; 
Stedman. 
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MEASURING THE HUMANITIES: THE SLIPPERY 
SLOPE FROM ASSESSMENT TO STANDARDIZATION 

MICHAEL HOLQUIST 


So scheint dcr komplizierte AuflosungsprozeB, dcr zur Zerfascrung 
dcr auBercn Handlung, zu BewuBteinsspiegelung und 
Zeitenschichtung flihrte, nach ciner sehr cinfachen Losung zu stre- 
ben. Vicllcicht wird sie allzu einfach sein fiir dicjenigen die unsere 
Epoche, trotz allcr Gefahren und Katastrophen, wegen ihres 
Lebensreichtums und des unvcrgleichlichen geschichtlichen 
Standorts, den sie bietet, bewundern und lieben. Abcr das sind nur 
wenige, und sie werden voraussichtlich von jener Vereinhcitlichung, 
die sich ankiindigt, kaum mehr als die ersten Anzeichen crlebcn. 

Erich Auerbach, Mimesis: Dargestellte 
Wirklichkeit in der abendlandischen Literatur (514) 

So the complicated process of dissolution which led to fragmenta- 
tion of the exterior action, to reflection of consciousness, and to 
stratification of time seems to be tending toward a very simple 
solution. Perhaps it will be too simple to please those who, despite 
all its dangers and catastrophes, admire and love our epoch for 
the sake of its abundance of life and the incomparable historical 
vantage point which it affords. But they are few in number, and 
probably they will not live to see much more than the first fore- 
warnings of the approaching unification and simplification. 

Erich Auerbach, Mimesis: The Representation of 
Reality in Western Literature (553) 


TOGETHER WITH CERTAIN TREES, such as California’s giant 
Redwoods, universities are among Earth’s oldest living things. The conceptual 
shape of universities has, of course, undergone enormous changes since their 
founding. If we take 1088 as the year of the institution’s origin 1 — when the 
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term “university” was first used in founding a new kind of school (“ universitas 
magistrorum et scholariurn ”) in Bologna — the academies we now so ruefully inhab- 
it go back for a millennium. And while the things called universities evolved 
differently in dilfcrent places, over the centuries there has been relative agree- 
ment about the nature of their task. They are seen as the final step in organ- 
ized education, the last opportunity cultures have to shape their future through 
systematic learning. As such, they have always been sites for disputes about the 
present and future course of the civilizations in which they operate. 

Revolutionary shifts in the history of the university as an institution are 
thus chartablc according to the crises that gave them rise. This brief essay is 
not the place to dwell on the periods of greatest transformation; suffice it to 
remember such relatively recent events as Napoleon’s defeat of Prussia in 
1806, leading to the von Stein reforms and the founding of Wilhelm von 
Humboldt’s University of Berlin in 1810.- Closer to home, consider the end 
of the War of Independence in 1789, and the debates that followed about 
what a distinctively American education should be, culminating in the Yale 
Report of 1828, the Morrill Act of 1862, and President Eliot’s reforms at 
Harvard in the 1 880s. After World War II, the several factors that produced a 
boom in higher education in the United States — pre-eminently the GI Bill — 
led to extraordinary changes in the American university. In the thirty years 
from 1945 to 1975, the number of undergraduate students in the United 
States increased by 500%, and graduate students by 900%, a demographic 
tsunami whose full effects we still do not fully grasp (Menand 144). 

Even more recently, enormous changes have swept through the most ancient 
of university systems as part of the effort to form a unified Europe following the 
end of World War II. Margaret Thatcher’s Kultiirkampf against Oxford and 
Cambridge during the 1 990s transformed higher education in the United 
Kingdom. And on the continent, “The Sorbonne Declaration” of 1 998 (signed 
by ministers from four European nations on the occasion of that university’s 
800th anniversary) sought sweeping changes. A year later, even more radical 
reform was called for in a document that was signed by twenty-nine European 
ministers of education (called by them — no doubt wishing to bask in the authori- 
ty of Europe’s oldest university — “The Bologna Declaration”). As of 2009, when 
its various national administrators met in Leuven, the Bologna program had 
grown even larger: it now includes forty-six nations (Council of Europe). 

Along with many others, then, I believe that in the first decades of the 
twenty-first century, global higher education is experiencing yet another revo- 
lution. It is one that shares certain structural similarities with past upheavals, 
but that has as well distinctive features of its own. Some of these portend even 
greater changes in the not too distant future. The current drive to institution- 
alize outcome assessments of various kinds in higher education is one of the 
more significant indicators of change at the moment. In this paper I seek to 
understand some of the unique implications of assessment in the humani- 
ties — particularly in the study of literature — as these relate to the institution of 
the university. 

While different kinds of crisis lay behind the many past transformations of 
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the university here and abroad, 1 these upheavals nevertheless had several fea- 
tures in common. They were first of all organized around different attempts to 
improve education, of course: a teleological red thread runs through all these 
arguments, organized as they are around the same question: “What is educa- 
tion for?” While there were differing views on this question going back to 
antiquity, beginning in the seventeenth century two opinions about the univer- 
sity’s mission crystallized as the polar extremes defining the debate — and they 
still do. The first comprised those who argued for a direct, seamless connec- 
tion between what universities teach and the specific careers that students 
enter immediately after graduation. That faction was opposed by another; its 
argument was that the university is a place where character and intellect are 
shaped in such a way that students attain the discipline and intellectual habits 
of inquiry required to be successful no matter what career they embark upon. 

To clarify and give some depth to the arguments of these two factions, I 
begin by invoking Kant’s diatribe against the University of Konigsberg in 
1796, which provides a particularly clear definition of both sides. Kant’s 
attack on his own university (where he not only taught, but served in 1786 as 
its Rektor) was directed against its medieval organization into Higher and 
Lower Faculties. Underlying Kant’s criticisms was a radical new conception of 
the human subject, one that required a novel kind of education. Wildly over- 
simplified, it might be said that for Kant, a reasoning person was a subject 
who constantly strove for autonomy in the face of society’s heteronomous 
claims on his individual character. 

In order to be a morally, as well as intellectually responsible and inde- 
pendently reasoning individual, the Kantian subject needed to develop the 
capacity for making his own critical judgments. Kant had argued in his three 
Critiques that thinking itself was essentially the capacity to make judgments 
( Urteilskrqft ). Criticism, understood as the mind’s activity in making judgments, 
was the very matrix of his whole system. Kant’s quarrel with the university in 
1796/8 was a practical application of metaphysical ideas he had proposed at 
least as early as 1781 in The Critique of Pure Reason. His argument for university 
reform might be summed up as follows: higher education (as constituted in the 
late eighteenth century) failed to foster the autonomy needed by students if 
they were to exercise the faculty of judgment they required to become respon- 
sible moral and intellectual agents. 

Specifically, Konigsberg was organized into a Lower Faculty that taught 
general subjects (history, philosophy, mathematics, and some science), and a 
Higher Faculty comprised of three professional schools in theology, law, and 
medicine. In drawing a distinction between the Lower and Higher Faculties, 
Kant pointed to the emphasis on training to think that characterized the for- 
mer and the stress on training for specific tasks in employment fostered in the lat- 
ter. Kant argued that something was missing in the schools whose primary mis- 
sion was to train people for jobs in the world outside the university: they all 
avoided foundational inquiry — not least of all about the very specialties they 
were teaching. However, as Kant saw it, this was as it should be: lawyers need 
to have confidence in the law, doctors to have competence in medicine, and 
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preachers should have faith in their doctrine. The state was correct to suppress 
any radical questioning of doxa in training for these professions, given the 
nature of the essentially vocational task with which they were charged. Such 
adherence to already available, practical knowledge was appropriate for what 
Kant somewhat maliciously termed “the businessmen ( Geschaftsleute )” of the 
higher faculties. 4 

But the Lower or Philosophical Faculty, Kant argued, had a different mis- 
sion: its task was ultimately to acquaint students with rational knowledge ireine 
Vernunfterkenntnisse) such as mathematics and philosophy, the metaphysics of 
nature and morals. Most significantly, the Philosophical Faculty studied the 
relation of metaphysical questions to the demands of everyday life, including 
the practical concerns of the professional schools. Thus, the Lower Faculty 
“extends to all parts of human knowledge” and can do so because it does not 
treat the subjects it studies as “its own content, but as objects it will examine 
and criticize for the benefit of the sciences” ( Conflict 28). 

In practice, this meant not accepting at face value claims the current state of 
knowledge took as self-evident. As Kant famously put it in the Preface to the 
first edition of The Critique of Pure Reason , “Our age is the genuine age of criti- 
cism, to which everything must submit” (100-101). And he specifically isolates 
Religion and Law as legitimate targets of such criticism: 

Religion through its holiness and legislation through its majesty 
commonly seek to exempt themselves from [criticism]. But in this 
way they excite a just suspicion against themselves, and cannot 
lay claim to that unfeigned respect that reason grants only to that 
which has been able to withstand its free and public examination. 
( 100 - 101 ) 

Fast forwarding to the present, my argument is that the university is once 
again best understood as a site of contest between the forces Kant contrasted 
as the Higher and Lower Faculties. Today it is the humanities that undertake 
the task Kant ascribed to the Philosophical Faculty. Understood as the liberal 
arts, their scope is pretty much what it was in Kant’s time. By contrast, the slot 
formerly held by the Higher Faculty-* has greatly expanded to include the vari- 
ous subjects that now claim the most students and funds: finance, economics, 
accounting, technology, and the softer social sciences. Many objections could 
be made to this admittedly over-simplified equation of the humanities with 
Kant’s Philosophical Faculty. My reason for nevertheless doing so is the con- 
viction that both the Lower Faculty at Konigsberg and the humanities in 
twenty-first-century American universities face much the same kind of threat 
to their capacity to exercise criticism across the boards. 

Kant’s conception of criticism was all encompassing; in his lifetime he was 
not known as the “All-crusher” (der Alleszermalmer) for nothing (Kuehn 246). 
Thus any criticism of his focus on criticism must perforce fall into the category 
of meta-criticism. For enemies of such radical questioning, such as the 
Prussian state, the safest tactic for combating criticism critically was simply to 
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declare an arbitrary end to the process, a kind of Schmitt-ian state of emer- 
gency in political logic. Which is precisely what happened when Friedrich 
Wilhelm II invoked censorship to counter Kant’s arguments. 

The situation is much more complicated, but no less threatening, in the 
twenty-first century. The divide between those who now hold the ideals Kant 
championed in the late Enlightenment and those who represent the values of 
the professional schools yawns even wider in our own day. Humanists, for the 
most part, continue to espouse the ideal of autonomy and cultivation of the 
individual person. Moreover, they continue to advance many of the same 
arguments put forward by Kant and von Humboldt. Crudely put, one might 
say they champion refinement (or better, Bildung ) of the student in preparation 
for the unpredictable trials of life, as opposed to training the student for a par- 
ticular job. Martha Nussbaum has been a compelling voice for this point of 
view, maintaining that a democracy is unthinkable without such training.® 
Steeped in the Greek classics, she goes back to fifth-century Athens to demon- 
strate that issues currently troubling us arose a long time before the university 
appears as an institution. Her argument is subtle, but Socrates’ searingly direct 
mandate to “know thyself” is still at its heart. 

It has frequently been pointed out that Socrates’ high-minded insistence 
on self-cultivation often devolved in practice into a tool of class privilege. 
Nussbaum counters such charges by invoking a variation on American excep- 
tionalism to insist that such has not been the case in the United States: “From 
early on, leading US educators connected the liberal arts to the preparation of 
informed, independent, and sympathetic democratic citizens” (“Skills for Life,” 
15). The many radical innovations that came into being during the career of 
Charles William Eliot as President of Harvard University for forty years (1869- 
1 909) may serve as a paradigm of the American attempt to use the liberal arts 
as a means of educating citizens for participation in a democratic society. 7 In 
the years following the Civil War, Eliot succeeded in opening Harvard to a 
broader population of students by abolishing the requirement that incoming 
students know Greek, introducing the elective system, and always insisting on 
the liberal arts as a major element in higher education. He specifically encour- 
aged the study of languages, literature, history and philosophy. 8 

How well his dream of the transformative power of a liberal education 
triumphed may be judged by the resurgence of the powers that emerged to 
oppose the classical liberal arts. A populist argument for vocational training 
emerged during the very years that Eliot was effecting his changes in the name 
of a people’s democracy at Harvard. Further south, Yale University’s philoso- 
phy of liberal arts education faced legal challenges brought by an association 
of yeoman farmers. It is a little known fact that when the Morrill Act became 
law in 1862, Yale — along with such other institutions as Iowa State University 
and Kansas State University — was among the very first six schools to benefit 
from it. That legislation was part of the American attempt to democratize the 
university by donating public lands to support, and in the words of the bill, 

“to teach such branches of learning as are related to agriculture and the 
mechanic arts” (Morrill Act). Unlike most other schools that benefited from 
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the Morrill Act, Yale had no farm. Nevertheless, it was the logical choice to 
become Connecticut’s designated land-grant institution, since there was no 
public university in the state in 1862. Further, unlikely as it now may sound, 
Yale in the mid-nineteenth century was a national leader in agricultural stud- 
ies. Sheffield, Yale’s scientific school, had already appointed this country’s first 
professor of the subject in 1846, and fourteen years later organized a national 
conference on agriculture that attracted 500 attendees and lasted a full month. 
The conference was widely covered by the national press. 

But you could see that trouble was already brewing for Yale. The New York 
Times reported its pleasure at the event, adding “We hope to see not only Yale, 
but all our colleges turning a portion of their attention from the languages of 
nations who have long since passed away, and directing it more to those truths 
of science” (qtd. in Schiff). In its coverage, Scientific America?! made the lan- 
guage/ science antagonism even more explicit: “To see Yale college stepping 
out from among the mists of antiquity and the graves of dead languages, and 
‘taking up the shovel and the hoe,’ is certainly one of the signs of the times” 
(qtd. in Schiff). 

The binary opposition of bad old languages vs. good new science reflect- 
ed in these articles did not fail to resonate with many Connecticut citizens. 
They felt Yale had not gone quite far enough to step out of the graves of dead 
languages. Among these were members of the very powerful National Grange 
of the Order of Patrons of Husbandry, a union of farmers then organizing 
across the country. They played a key role in the years after the Civil War, one 
difficult now to imagine in national and state politics. They resented what they 
called a “classical college” like Yale benefiting from the public largesse of the 
Morrill Act. It especially galled them that the three-year curriculum of the 
Sheffield degree in agriculture placed heavy emphasis on not just dead lan- 
guages from the mists of antiquity, but such modern languages as French and 
German. A Yale report on the agricultural school in 1 865 stressed that “the 
educated farmer should read [these languages] with ease” (qtd in Schiff). 

The Grange lobbied tirelessly, and in 1887 the Connecticut General 
Assembly voted to transfer its land-grant money to a state-run agricultural 
school. The Storrs Agricultural Station was established in western Connecticut, 
later to become the University of Connecticut. In the age of such agribusiness 
giants as Cargill and Monsanto, it would be easy to dismiss the views — and the 
political power — of nineteenth-century yeomen farmers as aberrations of our 
history. But the opposition the nineteenth-century Granges represented to 
“classical colleges” is rooted very deeply in American soil, and resistance to lan- 
guage study is still very much part of the American landscape. 9 

The conflict between the humanities and their more pragmatically orient- 
ed opponents has in the last twenty years taken on a new urgency. This radical 
shift in the debate is illustrated with frightening lucidity in reforms that have 
been introduced in the United Kingdom. Because major funding for universi- 
ties there comes from the government, you can see a process unfolding more 
slowly in the US with a clarity found only in governmental decrees. For the 
last twenty years, British universities have received funds through government- 
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administered Research Assessment Exercises (RAE). Under RAE, every five 
years universities submit requests for funding that are essentially competitive 
grant applications. Requests for funding must be supported by elaborate data 
that list faculty accomplishment, measures taken to improve the “research 
environment,” evidence of the esteem in which the university is held in the 
great world, and a number of other details. After review by panels of senior 
academics (sometimes including American specialists 10 ), scores are assigned 
and more or less money is provided to the competing institutions depending 
on the rating they have been assigned. 

Expensive, time consuming, open to manipulation and error as this proce- 
dure was, it had at least a patina of legitimacy derived from the role of the 
academic panels that assigned the scores. In the latest version of the review, it 
appears that the role of these academic panels has been reduced. There has 
been widespread questioning of the system over the years, culminating in the 
late Labor government’s decision to reform it. 11 This is still being worked out, 
and is complicated by Labor’s loss of the recent general election in the UK, 111 
but what seems to be emerging is a process still based on data concerning pub- 
lications by staff, research environment, etc., although the whole process has 
been given a new name: Research Excellence Framework (REF). Most notable 
among changes that have been introduced is a whole new category of compe- 
tition under the label “influence.” The creation of this arcane category becomes 
more understandable if you keep in mind that the Labor government transferred 
responsibility for higher education to the Orwellian sounding “Department of 
Business, Innovation, and Skills.” 13 

Not only are university programs to be judged in future on the murky cri- 
terion of “influence,” but a full 25% of the final score — on which their fund- 
ing for the next five years depends — will be based on it. The document 
announcing this change contains 

a “menu” of “impact indicators” that will be accepted: it runs to 
thirty-seven bullet points. Nearly all of these refer to “creating 
new businesses”, “commercialising new products or processes”, 
attracting “R&D investment from global business”, informing 
“public policy-making” or improving “public services”, improv- 
ing “patient care or health outcomes”, and improving “social 

welfare, social cohesion or national security” Only five of the 

thirty-seven bullet points are grouped under the heading 
“Cultural Enrichment”. These include such things as “increased 
levels of public engagement with science and research (for exam- 
ple, as measured by surveys)” and “changes to public attitudes to 
science (for example, as measured by surveys).” The final bullet 
point is headed “Other quality of life benefits”: in this case, 
uniquely, no examples are provided. The one line under this 
heading simply says, “Please suggest what might also be included 
in this list.” (Collini) 
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It is clear, from language used in the document, that the government 
expects that the criteria forjudging any given university’s “impact” should be 
uniform across disciplines. Whether they give degrees in science, business, or 
humanities, programs must adhere to the same definition of what constitutes 
their relevance. The framers of this policy conceive “impact” as defined by 
results that are “closer to the market” (qtd. in Collini). Keith Thomas has 
recently called attention to a document entitled A Vision for Research, in which 
the Prime Minister’s Council for Science and Technology recommended that 
“universities should seek to professionalize their capabilities and structures . . . 
so that they operate more like consultancy organizations” (13). A memoran- 
dum from King’s College London makes clear at the level where the rubber 
meets the readjust what this means: due to financial exigency, it is necessary 
to “create financially viable academic activity by disinvesting from areas that 
are at subcritical level” (qtd. in Thomas 1 3). 

It is, of course, the humanities that administrators who publish such mem- 
oranda have in mind when they inveigh against “areas that are at subcritical 
level,” and British universities have seen spectacular declines in literature, his- 
tory, and foreign languages. At King’s College London, “arts and humanities 
faculty ... are being forced to reapply for their jobs. When the evaluation is 
finished, around twenty-two of them will have been voted off the island” 
(Grafton 32). 

I have concentrated on what is happening to the humanities in the UK 
because the global profile of the threat facing the humanities — and pre- 
eminently literary scholarship — is harder-edged and easier to see there than it is 
here in the US. The situation in the rest of the world is equally loaded in favor 
of practical training — what is called STEM education (Science, Technology, 
Engineering, and Mathematics), as anyone who has visited Chinese, Japanese, or 
Indian universities is aware. 14 And as Keith Thomas has recently reminded us, 
the highly influential Shanghai Jiao Tong world ranking of Universities “is 
based on criteria which take no account of the humanities at all” (15). 

The European continent — where the vision of the humanities arose that 
shaped the liberal arts in the United States — is currently convulsed with rolling 
waves of change and displacement in its universities. The Bologna Process has 
spread like a plague, to the consternation of students who now make less use 
than ever of the admirable ERASMUS program designed to ease transfer of 
credits between national university systems. In a recent poll, only four percent 
of European undergraduates took advantage of ERASMUS (cited in 
Adclman, Bologna Process xxii). In Germany last year, under banners bearing 
quotations from Goethe (Nicht Amboss, sondern Hammer sein [Be the hammer, 
not the anvil]), tens of thousands of students marched, rallied, and mounted 
protest sleep-ins in over fifty cities to dramatize their resistance to the 
Bologna reforms. ^ 

German professors have been particularly vocal in their rejection of the 
Bologna Process. In 2009, the distinguished theologian Marius Reiser resigned 
from his post as Professor at the Gutenberg University in Mainz to protest the 
reforms. 111 And Julian Nida-Rumelin, an eminent political scientist at the 
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Scholl Institut of Munich University and former Minister for Cultural Affairs 
under Gerhard Schrocdcr, 17 has repeatedly spoken out against the Bologna 
Process. The main objections to the Bologna Process include the infantilizing 
effects of the reforms (they turn universities into secondary schools); the per- 
verse negative effect on student mobility of standardizing curricula across 
Europe; 18 and the time wasted by students memorizing texts (hardly any time 
remains for studying alone). At least in the humanities, “this is — to put it mild- 
ly — regrettable” (Nida-Rumelin, Bologna Process). Or, put another way, “... the 
Bologna Process is all about teaching of skills [K'enntms ] — in other words about 
giving students basic knowledge and an overview of a subject. Everything is 
aimed at vocational training, universities providing skills that students will later 
need in their jobs” (Reiser). 

Looming over these criticisms is the shadow of the systemic feature that is 
meant to be the force able to insure standardization across national borders. I 
mean of course the enormous mechanism of assessment procedures required 
to guarantee the sameness of outcomes across 46 countries, over 4,000 univer- 
sities, and 16 million students speaking 23 languages in the 27 countries of the 
European Union alone (Jaschik, “Tuning”). From the outset, the Bologna 
Process has wrestled with increasingly elaborate schemes of assessment in 
order to address the problem of how to homogenize such multifarious parts. 
And those parts are more disparate than is usually supposed: contrary to pop- 
ular belief (in the US, at least), the Bologna Process is not merely a European 
phenomenon: it does indeed include members of the European Union, but it 
now also comprises such outliers as Azerbaijan, Kazakhstan, and Georgia. 
With the spread of the system, the problem of how to achieve outcomes that 
can be meaningfully compared continues to grow. As one official of the 
Bologna Process officials said to me, “It’s like herding cats!” 19 

I invoke the Bologna Process because it brings into highest relief the con- 
nection between assessment and standardization. That is its utility for Americans: 
you can see an aspect of assessment that is frequently overlooked in the less 
hegemonic attempts to promote outcomes assessment in US universities. In 
most American discussions, standardization — a constantly lurking danger in 
any assessment program — frequently gets downplayed or goes unacknowl- 
edged. It is often dismissed as an aberrant effect of the No Child Left Behind 
(NGLB) agenda that has had disastrous results for American primary and sec- 
ondary schools," and thus not a matter of concern for professors in higher 
education. I believe this is naive. It is not just parents who must mortgage their 
homes to pay ever-rising tuition fees who wish to see what they are getting for 
their money. There is no reason to believe that the forces inside government, 
big business, and the extreme populist/ authoritarian segment of the popula- 
tion at large who spearheaded NGLB will fail to see the historical independ- 
ence of the American university as a potential danger or non-systemic affront 
to economic rationalization. 

Gerald Graff has famously written that “Assessment Changes Everything,” 
and indeed it does — especially during a financial crisis. Some of the things 
that get changed in the push to introduce assessment procedures in American 
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higher education are relations between universities and state and federal gov- 
ernments. I suspect one reason assessment’s inevitable effect of standardiza- 
tion has played a less prominent role here than it has in Europe is the comfort- 
ing assumption many US academics make that such reviews would be exclu- 
sively peer reviews, designed and administered by those to be assessed. That is, 
many American academics think that, unlike Europe, where provincial and 
national governments run the universities and thus oversee their assessment, 
the idiosyncratic American tradition of private colleges and universities would 
insure that assessment will be equally autonomous. And up to this point, the 
most far reaching attempts to introduce assessment have been the creation of 
academics themselves, as in the admirable program instituted by the office of 
academic planning and assessment at the University of Massachusetts at 
Amherst*^ 1 and many other places. 

However, there are those here at home in the United States who seek to 
expand the reach of assessment into a much more far-reaching system. At the 
January 2009 meeting of the Association of American Colleges and Universities 
(AAC&U), two national education campaigns were announced, both of which 
effectively work to intensify and standardize assessment in US higher educa- 
tion. I wish to make very clear that neither of these organizations has an offi- 
cial, governmental status. But they are now the most active proponents of 
assessment on a broader scale than has been traditional in American higher 
education. A National Institute for Learning Outcomes Assessment (NILOA) 
was initiated to “assess assessment” at that January meeting. The creation of a 
New Leadership Alliance for Student Learning and Accountability was also 
announced Jaschik, “Assessing”). 22 The impulse behind these moves — and 
many others of the sort — is best summed up as “the best defense is offense.” 

As Molly Corbett Broad, president of the American Council on Education, 
put it at the AAC&U meeting, higher education in the US “cannot be playing 
defense. That is the message of the day” (qtd. in Jaschik). 

And the two new organizations that sprang into being on that occasion 
appear to be well advanced in their activity. NILOA has prepared a question- 
naire seeking information on current assessment practices and distributed it to 
the presidents of every two-year and four-year college and university in the 
country. Furthermore, conferences have been organized, attended by officials 
from Minnesota, Indiana, and Utah, to discuss the merits of the Bologna 
Process. 

What I find somewhat unsettling is that these groups frequently work with 
two others, the Institute for Higher Education Policy (IHEP), and the Lumina 
Foundation. Both of these think tanks (and funding sources) have been push- 
ing the Bologna Process as a model for Americans to follow. They have been 
spurred by Clifford Adelman’s white paper, The Bologna Process for U.S. Eyes: Re- 
Learning Higher Education in the Age of Convergence. Arguing from a militandy glob- 
alist perspective, Adclman (who before joining IHEP worked at the US 
Department of Education for over thirty years), is trying to wake America up. 
With Lumina Foundation support, he is currently directing IHEP’s “global 
performance” project, for which his glowing version of the Bologna process 
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was prepared, and which also sponsored an earlier paper of his, Learning 
Accountability from Bologna: A Higher Education Policy. Adclman makes no bones 
about the fact that he considers the voluntarist efforts of American colleges 
and universities to be virtually useless, making clear that he strives for a system 
like Europe’s much more far-reaching and centrally organized system: 

The “voluntary system of accountability” adopted by a large seg- 
ment of higher education — which tells the public how many 
pieces of paper colleges and universities handed out (to whom 
and when), how much students liked different aspects of their 
experience at an institution, and how much scores on tests of 
something called “critical thinking” improved for a sample of stu- 
dents between entrance and senior year — is more show than sub- 
stance. (Learning Accountability 1) 

In Adelman’s account, the United States is portrayed as lagging behind the 
rest of the world because Americans do not pay enough attention to develop- 
ments overseas: “Former Secretary of Education, Margaret Spellings’ 
Commission on the Future of Higher Education paid no attention whatsoever 
to Bologna, and neither did the U.S. higher education community in its under- 
whelming response to that Commission’s report. Such purblind stances are 
unforgivable in a world without borders” (Bologna Process viii). Now, he surely is 
right about US ignorance of what is happening in higher education in the rest 
of the world. But it does not follow that we should therefore do more to inform 
ourselves in order to imitate programs that have yet to prove themselves in 
Europe, such as the Bologna Process. As the data I have cited above make clear, 
there has been widespread controversy, to say the least, over the worth of that 
process from both European students and professors, while its strongest propo- 
nents have been governmental functionaries in every nation where it has been 
implemented. And anyone who has read Diane Ravitch’s last book, The Death 
and Life of the Great American School System: How Testing and Choice are Undermining 
Education , will find Adclman’s use of the term “accountability” chilling. 

I, too, deplore American ignorance about changes in university educa- 
tion across the world. With regard specifically to the Bologna Process, it 
seems to me that, far from being a model, Americans might well see Bologna 
as a warning about what might happen here. I have not (yet) found examples 
quite as egregious as those I have cited in Germany and the UK," 3 but every- 
where across the United States there is an increasing pressure to do what the 
Europeans already have done: one, to downsize the same amount of learning 
into a shorter period of time (a three-year BA)" 4 ; and two, to sacrifice all 
other goals in the service of standardized outcomes across the board, no mat- 
ter what the effect might be on different areas of scholarship, university sys- 
tems, or individual students and professors — the humans who are the subject 
of the humanities. 

National economic growth has replaced nationalism as the force organiz- 
ing universities in the twenty-first century."^ This is only one effect of a more 
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universal mercantilism, but since universities continue to be the pinnacle of any 
education system that has them, the commercialization of higher education will 
have more far-reaching effects in them than in other areas of our civilization. 

Since this is the case, it is not unreasonable to expect that every academic 
specialty be able to give some account of itself. After so much is granted, how- 
ever, there are several questions that still need to be answered: what is the 
nature of that account to be; who should provide it; to whom should it be 
addressed; and who should evaluate it? 

If the common sense program for which Gerald Graff has become a 
prominent spokesman were to be the furthest extent of any future assessment 
program, one might have little reason to fear the future. He essentially asks for 
more interaction between colleagues so as to rationalize the curriculum, thus 
giving students an education that is progressive, beginning at a lucid A and 
ending at a coherent Z. Departments should critically examine their joint 
offerings as a joint structure with an Aristotelian beginning, middle and end. 
Although most of us remember most vividly the odd course from our youth 
taught by a colorful eccentric, no system can afford too many of these. Who 
would disagree with Graff’s call for more cooperation among colleagues and 
his opposition to patchwork programs of stand-alone courses that are merely 
monuments to the pedagogical dervishes who teach them? Clearly, for col- 
leagues to gather and compare notes on the way each is contributing to a 
common goal that has been established for their degree is a good thing, and 
can actually enhance collcgiality if organized properly. This is a dialogue we 
should have been having all along. 

But beyond these workaday measures aimed at improving the curriculum 
so as to give students a clear, graduated sense of the particular sector of 
knowledge we profess looms a steep and slippery slope. There, a host of 
issues awaits us that raise serious questions about the applicable scope of 
assessment. Consider: assessment of any kind is essentially a judgment — it 
has the same root as assizes — with at least two necessary components: the 
thing to be assessed, and the scale that will be used to assess that thing. For 
assessment to work, there must be a fit between these two elements: you can- 
not measure time with a ruler, or space with a clock. So we must ask: what is 
the proper instrument and what is the correct unit of measurement appropri- 
ate to assess courses in the humanities? 

No one yet has come up with compelling answers to these questions. It 
seems evident that — if we are not bent by the winds of insane standardizat- 
ion 211 that sweep through the Bologna universe — we must resist attempts to 
measure what we do when we teach Proust or Dostoevsky through glorified 
time and motion studies calibrated to encourage ever greater “impact” ratings. 
Such controls are still mostly a future threat in the United States, but at a time 
when influential forces here at home are holding Bologna up as a model to be 
imitated, we should heed the warnings coming from Europe where they are 
the norm. This is not xenophobia. As I tried to indicate earlier, Europeans 
themselves — including those tasked with implementing the reforms — have 
raised questions about what they are doing. In May 2010, the European 
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University Association itself published a white paper entitled Impact of the 
Economic Crisis on European Universities , which concludes with a warning: 

Finally, there are indications that the economic crisis has also had 
an impact on the development of institutional autonomy. Where 
authorities resort to direct steering mechanisms, regulations and 
increasingly unbalanced accountability procedures, there will be 
counterproductive effects, preventing universities to fully act as an 
essential player in overcoming the crisis. (4-5) 

The same economic crisis impelling reform in Europe has lent the cam- 
paign for assessment in higher education new weight and urgency in the US. 
And, as in Europe, when administrators are faced with the need to make 
budget cuts, the first areas affected are always in the humanities, and particu- 
larly in foreign languages. Elimination of such programs has become so com- 
mon a feature on our horizon that even concerned members of the Modern 
Language Association (MLA) now hardly take notice when yet another depart- 
ment bites the dust. Items such as the following are weekly to be found: “This 
is not an easy time for foreign-language departments. Programs at California 
State University at Fullerton and the University of Maine at Orono, to name 
two, were recently shrunk, and decisions about the fate of some language pro- 
grams at the University of Nevada at Reno and University of Tennessee at 
Knoxville are pending” (Miller, “Foreign-Language”). 

Such closures are not only sad; they contain an irony and a warning. The 
irony is this: of all the disciplines in the humanities, none has done more than 
the foreign languages to — voluntarily — introduce assessment procedures. 

Their track record is a model of establishing — and tracking — norms. Of all 
humanistic disciplines, foreign language departments come closest — and have 
done so before the issue was recently politicized — to meeting the expectations 
of assessment enthusiasts. 

There are many who argue that if humanities departments just came up 
with their own assessment procedures, they would be spared having norms 
imposed on them from outside. Against this premise stands the lugubrious his- 
tory of foreign language departments in this country: it is precisely they who 
historically have given the strongest support to assessment that are now among 
the most endangered. 

Twenty-five years ago, the American Council on the Teaching of Foreign 
Languages (ACTFL) introduced a program of Certified Proficiency Testing 
that accurately assesses any subject’s ability in many languages (65 tests for 
oral proficiency, 12 for written). These assessments are so well designed and 
transparent that the Department of Defense counts on them. Nevertheless, the 
lights in foreign language department offices are going out all over the land. It 
is clear that establishing clear-cut standards on which accurate assessment of 
student knowledge can be based is not in itself a weapon strong enough to 
overcome resistance based on brute economics. Unless some way is found to 
protect foreign language teaching from the blind sword of “impact,” this 
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country is going to lose strength in the very sinews of that globalism now so 
universally embraced as a value — the languages in which others think and 
conduct their emotional lives. 

The situation becomes more involved when we introduce literature cours- 
es into the equation of relevancy. For all the good it has done them, foreign 
language departments can demonstrate they have already done what is being 
asked of us by assessment enthusiasts. They have been able to do so because a 
certain testable precision is ineluctably part of what they do: it’s either “amo, 
amas, amaf or not, right? Basic language learning is a skill acquisition. Learning to 
speak French in college is similar to other courses in which you learn specific 
skills: in accounting courses, the books balance or not, right? It is no denigra- 
tion of basic language teaching — an excruciatingly difficult task performed by 
dedicated teachers who are experts in their own right — to acknowledge that 
studying literature is not only not like studying accounting; it is not even like 
French 101. 

In foreign language departments, as in English departments, the classes in 
which literature is taught are engaged in something different from getting stu- 
dents to memorize declensions (although such literature courses do, of course, 
at the same time add to students’ language skills). But what that something 
else is, is difficult to define in hard-edged terms of a kind that are easily 
amenable to assessment procedures. 

Most will immediately accede to the proposition that what goes on in a lit- 
erature course is different from what goes on in a first-year language course. 
But once so much has been said, it is difficult to calibrate further the differ- 
ence in what is learned in each. What can be said with certainty is that after 
our first halting steps, whether in our native tongue or one we acquire later in 
life, language becomes more than a simple skill set as it evolves into the very 
means by which we think. The language in which we argue with others or 
meditate in our own minds as we mature becomes something more than the 
mere knowledge of vocabulary, grammar, and syntax. It is the fuel that powers 
the engines of our very minds. And it is language at that mature level that the 
exercise of reading critically in literature courses fosters and enriches. 

In basic language courses one acquires information , while in literature 
courses (no matter in what language) one acquires the habits of critical 
thinking in the Kantian sense that can — not necessarily, but possibly — lead 
to what used to be unabashedly called wisdom. When mercantilist critics 
call for universities to become more like high schools or trade schools, they 
demonstrate ignorance of possible levels of knowledge that are not mere informa- 
tion. The best definition of such information still belongs to Walter 
Benjamin (who sees it as the kind of knowledge at the other end of the 
spectrum from wisdom)'. 

The value of information does not survive the moment in which it 
is new. It lives only in the moment; it has to surrender to it com- 
pletely and explain itself to it without losing any time. (90) 
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Taylorism is dead, but the efficiency expert impulse to measure precise 
bits of output and impact data is, zombie-like, still very much with us. The 
demand for such data is beginning to come not just from high-minded, self- 
motivated faculty and administration in our colleges, nor from foundations 
such as Lumina, or think tanks such as IHEP. As in Europe, government is 
stepping into the picture. The state of Texas has long been notorious for its 
governmental control over what is taught in its primary and secondary 
schools, but it now has legislated a measure of unique prurience to insure that 
diktats of the state are obeyed in its colleges and universities. The Texas 
House and Senate — in a move Governor Rick Perry called a contribution to 
“consumer protection” — has unanimously enacted a new law that will force 
everyone involved in higher education in Texas to post on a public website 
specific, detailed information about their classroom assignments, syllabi, curric- 
ula vitae , department budgets, and the results of student evaluations. None of 
the information can be more than three clicks away from the college’s home 
page, and each college or university must have a compliance officer who will 
send a written report to the governor and legislative leaders every other year. 27 

This law reflects the continuing influence of the former chairman of the 
Commission on the Future of Higher Education, Houston investment consultant 
Charles Miller, a heavy contributor to the campaigns both of President Bush 
and Texas governor Rick Perry. While chairing the Commission, Miller flood- 
ed its members with a number of “issue papers,” some written by himself, and 
others written by allies, such as Robert C. Dickerson, a former vice president 
of the Lumina Foundation. 28 One of those papers, written in 2006 (i.e., four 
years before the Texas law was passed), is an explicit call for oversight of the 
kind Texas House Bill 2504 succeeded in making a legal requirement. In an 
“issue paper” entitled Accountability /Consumer Information, after calling for infor- 
mation of the type contained in the Bill, Miller writes: 

Any number of excellent consumer shopping sites could serve as 
models for the revised college search site. While shopping for a 
postsecondary institution is not exactly the same as shopping for a 
car, many on-line shopping sites embody extensive flexibility that 
allows consumers to specify their needs and interests and to com- 
pare products that meet criteria set by the consumer. (2) 

Texas is, as they say in the Austin state house (modeled on the capitol 
building in Washington, “but with a higher dome”), something of a maverick 
among state legislatures. What HB 2504 makes clear is that governmental 
oversight can trump whatever local efforts of well-meaning faculty at Rice 
University or the University of Texas might have made to institute their own 
assessment program. The larger point is that attempts to initiate assessment 
programs from within the academy should be made, but they should always be 
made with an eye to the external environment in which they are made. 29 

House Bill 2504 is disturbing not only because it actualizes the possibility 
that assessment procedures in higher education can be mandated by govern- 
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ment, as they have been in K-12. The rhetoric surrounding HB 2504 — “con- 
sumer protection,” “on-line shopping,” “criteria set by the consumer,” etc. — 
indicates that “while shopping for a postsecondary institution is not exactly the 
same as shopping for a car,” it is similar enough to be cast as a commodity, 
and like all commodities, it must shape itself to the desires of the market. The 
obvious lesson to draw from this danger is that we must find ways to go about 
our task of trying always to improve our work, but must do so guided by the 
standards of our profession, as well as the prejudices and fashions of the mar- 
ket. Another way to conceive the slippery slope we are on is to see yawning at 
its bottom the specter of American mercantilism. 

What I am saying is that there are two components of assessment that 
must be given greater attention in our attempts to improve what we offer our 
students. The first of these is accountability. As Diane Ravitch makes dreadful- 
ly clear, assessment can all too easily be conflated with a system of rewards and 
punishments. If criticism were guided by intellectual goals of Kantian purity, 
this might not be such a bad thing. Education is inherently not democratic, and 
when we give grades we all use assessment as a means to discriminate among 
different levels of quality. But when influential folk like Charles Miller wish to 
turn education into a commodity whose shape is determined by consumer 
demand, or when the administrators of King’s College London argue that it is 
necessary to “create financially viable academic activity by disinvesting from 
areas that are at subcritical level,” they are using assessment as a weapon to 
force education into their unabashedly un-academic vision of what it should 
be. And by taking the financial bottom line as the academic bottom line they 
will predictably eliminate idiographic programs in the humanities. 

In presenting our assessment designs to administrators at our home cam- 
puses, it would be useful to remind them what the world would look like if the 
capacity for deep reflection at the heart of philosophy, or the possibility of liv- 
ing other lives in fiction in our literature courses were eliminated. And such a 
lugubrious vision is now harder-edged than ever before. You can see what the 
commercialization of American higher education would produce in the future 
if you look at the phenomenon in the present of schools that are not just run 
like businesses, but are businesses. I am referring, of course, to the self-confess- 
cdly for-profit schools that have sprung up in the last two decades in the 
United States. For profits now enroll 3.2 million students — 1 1.8 percent of the 
27.4 million currently in college (Lederman). The largest of them, the Univ- 
ersity of Phoenix, has on its 200 campuses (and the Internet) the largest stu- 
dent body of any university in the US: 420,700 undergraduates and 78,000 
graduate students (“University of Phoenix”). John Sperling, the billionaire ex- 
professor from San Jose State who runs its parent, the Apollo Group, is one of 
Forbes Magazines’ richest men in the world. He is quoted as saying the University 
of Phoenix is “a corporation, not a social entity. Coming here is not a rite of 
passage. We are not trying to develop . . . [students’] value systems or go in for 
that ‘expand their minds’ bullshit ...” (qtd. in Washburn x). This is refreshing. 
Most of the people who are involved in administration of schools that are offi- 
cially non-profit would be embarrassed by Sperling’s candor. But the University 
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of Phoenix nevertheless exists as the Platonic idea of the university conceived 
exclusively as a business enterprise, where fiscal policy trumps academic 
responsibility: as Sperling says so well, “a corporation not a social entity.” 

Sperling’s candor would be amusing were there not an increasingly pre- 
cipitous slide on the slippery slope between Tor-profits and non-profits. You 
can see the boundaries shifting in a recent imbroglio involving the august 
Higher Learning Commission. The Inspector General of the US 
Department of Education has publicly criticized the work of the North 
Central Association of Colleges and Schools, one of the six regional accred- 
iting organizations recognized by the federal government, and the major 
regional agency accrediting colleges and universities in the upper Midwest, 
including the University of Michigan and Northwestern University among 
others. The Education Department’s major complaint concerned the over- 
sight agency’s leniency toward American Intercontinental University^ (a 
for-profit institution). According to the Department of Education, the major 
entity for evaluating colleges and universities in the upper Midwest had no 
‘“established definition of a credit hour or minimum requirements for pro- 
gram length and the assignment of credit hours,’ a situation that could lead 
to ‘inflated credit hours, the improper designation of full-time student status, 
and the over-awarding’ of federal financial grants” (qtd. in “Education 
Dept.”). I note in passing that if the highest level of official institutions 
charged with assessment of universities cannot agree on “established defini- 
tions” of such basics as credit hours, some slack should be given individual 
departments struggling with the rigors of assessment. But what is most trou- 
bling about this news item is that it shows the ability of for-profit universities 
like American InterContinental University to benefit^ 1 from decisions made 
by official agencies also charged with oversight of non-profit schools, some of 
them very prestigious indeed. Texas proponents of assessment by governmen- 
tal decree arc driven by the idea that an education is simply a commodity; thus 
it is not surprising that institutions have sprung up that offer degrees that are 
very little but commodities (and at public expense — the for-profits would go 
out of business tomorrow without the federal government’s annual expendi- 
ture of ®25 billion in grants and 585 billion in subsidized loans) (Basken). 

I adduce the scandal of the for-profits because they represent another ver- 
sion of what the bottom of the slippery slope of assessment looks like. In a 
better world, there would be a separate agency to accredit for-profit institu- 
tions, run, as they are, on principles different from the non-profits (although 
increasingly less so, it must be said). 

What can we conclude about assessment in the humanities from this 
irresponsibly brief overview of history, the current spasm of reform in 
European universities, and from the precipitous rise of for-profit institutions 
in the US? First of all, that we should be aware of the scope and power of 
the forces we are up against. For some time now, academics have suspected a 
commercial juggernaut is flattening traditional values, the ideals and princi- 
ples that have inspired universities since their inception. What was only 
sensed in an inchoate way in the past has now materialized into an all too 
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hard-edged reality. Outcomes assessment is a potentially useful tool that — 
unless we are very canny — may be transformed into the Trojan horse of 
those outside the academy who wish to turn education into a business, pure 
and simple. The list of those who wish to do so is long: the British treasury 
officials who oversee the REF, the European Community bureaucrats who 
control the Rube Goldberg machinery of the Bologna Process, and the 
nakedly greedy entrepreneurs of higher education for-profits such as the 
University of Phoenix in the US. They all have the institution of the univer- 
sity as a source of questioning and self-exploration in their sights. The Great 
Recession, the global financial collapse in 2008, put a powerful weapon in 
the hands of those marching under the assessment/impact banner, for it 
gave their demand for accountability real world urgency. 

And that is the great dilemma of the humanities in general and of the liter- 
ary humanities in particular. The distinction made long ago by the Neo-Kan tian 
historian of thought, Wilhelm Windclband, between the work of the humanities 
and the sciences is more relevant today than ever. His argument was that the 
human sciences seek to understand what is distinctive, unrepeatable, new and 
unpredictable in our individual lives and in human history. At the other end of 
the spectrum is the emphasis of the exact sciences to elicit from nature general 
laws, whose validity is greater the more abstract and predictable they are. These 
extremes — the idiographic humanities and the nomothetic sciences — are not 
mutually exclusive, of course. But they do point to the difficulty of defining the 
results of humanistic study, focused as it is precisely on uniqueness, in a way that 
is easily measurable. The relatively modest goals outlined by units such as the 
department of comparative literature at the University of Colorado at Boulder 
probably come as close as we can to finding assessable categories that are mean- 
ingful, without falling into a standardized straitjacket. 

But in the end, it is only in the scale of a whole lifetime that the worth of 
literary education may be measured, and that is a scale that cannot be 
“tuned.” When a tape measure or stopwatch or metronome is invented that 
will succeed in making the study of literature precisely measurable, it will have 
become another thing. If that happens, our children will look back at what it 
was and weep for its loss. 

I would like to thank my friends Dick Ohmann and Jerry Graff, and the co-editor of this 
volume, Laura Rosenthal, for helpful criticism in preparing this essay. Cheryl Clung, of the 
Teagle staff, provided good advice in the paper’s final stages. They are, of course, in no way 
responsible for the less than dianoetic excesses that remain. 


NOTES 

1 As my ignorance forces me to do, thus ignoring claims from such other 
parts of the world as India’s Nalanda University, or the great linguist 
Panini’s Takshashila (fourth century BCE), now in Pakistan, or the Islamic 
centers of learning such as Cairo’s A1 Azhar, the Qarawiyyn Mosque in 
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Fez, or Mali’s University of Timbuktu. 

~ Napoleon’s victory at Jena was only the effective cause of radical reform 
in Prussian education. A more likely proximate cause was Kant’s 1796 
tirade against the medieval organization of the University of Konigsberg, 
about which more is discussed in the essay. 

^ By “abroad,” I really include only Europe in any significant depth in this 
paper, denied by my ignorance of events in the rest of the globe from 
making further comparisons. Even from my parochial perspective, howev- 
er, it is clear that both China and India are pouring vast treasure into 
their university systems, but this is a tale very much still in the telling. In 
the case of China, there seems to be a developing trend that indicates, 
ironically, that as the West begins to eliminate humanities programs, the 
Chinese are strengthening them. I cite as evidence Wang Ning’s program 
in Comparative Literature at Tsingtao University (widely known as the 
“MIT of China”); the return (from his previous post at Duke) of Liu 
Kang to Shanghai Jiao Tong to become Dean of the Institute of Arts and 
Humanities; and the self-consciously liberal arts-oriented curricular 
reforms at the prestigious University of Hong Kong led by Amy Tsui (see 
Hennock, “University of Hong Kong Looks to West in Curricular 
Design”). The impulse to strengthen the humanities is a pan-Asian phe- 
nomenon: on September 15, 2010, the foundation of a whole new liberal 
arts college to be jointly administered by the National University of 
Singapore and Yale University was announced (see Caplan-Briker, “Yale, 
Singapore plan new liberal arts college”). 

^ German idealists were unusually inventive in coining derogatory terms for 
their enemies. Kant’s great admirer, Schiller, who spent years as a professor 
of history at Jena, dismissed his colleagues who concentrated on practical 
training as “ Brotgelehrte ,” or “bread-scholars” (i.e., people who studied mere- 
ly to prepare for a job to earn their daily bread), in his inaugural lecture, 
“What Is, And To What End Do We Study, Universal History?” in 1789. 

^ The Conflict of the Faculties has recently been invoked by a number of others 
as an aid to understand the present, most notably Pierre Bourdicu, in 
Homo Academicus; Bruce Robbins in “Less Disciplinary than Thou: 
Criticism and The Conflict of the Faculties”; and Jacques Derrida in 
“Mochlos; or, The Conflict of the Faculties,” and in one of his last lec- 
tures, “The University Without Condition.” 

® See Nussbaum’s Cultivating Humanity: A Classical Defense of Reform in Liberal 
Education and Not for Profit: Why Democracy Needs the Humanities, excerpts 
from which appeared as “Skills for Life: Why Cuts in Humanities 
Teaching Pose a Threat to Democracy Itself.” An even more eloquent 
argument has been made by the former Dean of the Yale Law School 
(who also has a PhD in philosophy) Anthony Kronman in Education’s End: 
Why Our Colleges and Universities Have Given Up On The Meaning of Life. It is a 
sign of the times that this book has not had a wider impact. 

^ Before becoming Harvard’s youngest president, Eliot had been a chem- 
istry professor. 
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The only real defeat he experienced occurred when he tried to abolish 
intercollegiate sports. 

^ Information on the Yale example is taken from Judith Ann Schiff, “When 
Yale Was a Farming School.” One indicator of the continuing struggle 
between the humanities and the sciences was the attack on literature pro- 
fessors at Yale made by no less than the then-dean of Yale College to the 
conservative National Association of Scholars in 1991. In his remarks — 
part of a symposium sponsored by the editors of Academic Questions — the 
then-dean charged that at Yale “the woods are full of humanists who 
doubt [‘the efficacy of reason, or the possibility of truth...’]” and that 
“people in the humanities . . . disagree about what’s good and bad, and 
overall seem a very funny bunch of guys” (“Doing the Right Thing” 34). 
Following the largest meeting of the Yale College Faculty ever held (initi- 
ated by the author of the present article) on October 3, 1991 at which the 
Dean’s remarks were discussed, changes were made in the administration 
in the following year. 

I® Full disclosure: the author of the present essay served on some of those 
RAE review panels. 

1 1 The RAE model has been influential in formulation of other European 
efforts to assess humanities scholarship. Consider the heated resistance of 
scholars to the European Reference Index for the Humanities (ERIH), the 
program sponsored by the European Science Foundation (ESF) to give 
grades to professional journals, ranking them according to whether they 
fall into category A (“high-ranking international publications”), category B 
(“standard international publications”), or C (journals “with important 
local/regional significance in Europe”). The goal of this effort is to be the 
“first step towards the development of a framework that will enable 
Humanities excellence to be assessed and verified” (qtd. in Smith). 

Sixty journals in history and philosophy of science published in their first 
2009 issue a joint editorial condemning ERIH as an initiative “entirely 
defective in conception and execution” and whose likely effect is that “we 
will sustain fewer journals, much less diversity, and impoverish our disci- 
pline” (Leiter). A particularly thoughtful response to the ERIH initiative is 
“Failing the Grade: The Craze for Ranking Humanities Journals” ( Kritika 
Editors). 

1 - But many in the British Academy fear things might be even worse under 
the new coalition government of Conservatives and Liberal Democrats 
headed by David Cameron and Nick Clegg that formed after May 6, 
2010. 

1 

10 For a full account of these developments, see Thomas, “What Are 
Universities For?” 

14 The situation seems to be somewhat changing in China. See note 3 in 
present paper. 

1 4 For an overview of earlier German resistance to Bologna, see the inter- 
view with Ulrich Tcichler, “Das Marchen vom Forschcrparadies.” It is 
somewhat unsettling to discover that from at least the early modern peri- 
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od, German governments were seeking to assess the performance of pro- 
fessors, even to the point of sending spies around the state’s institutions of 
higher learning: “In 1569 the elector of the Palatinate, who was the 
patron of the University of Heidelberg and sovereign over the land, 
ordered the university to send a report containing the names of the pro- 
fessors, what topic and at which time each professor lectured, and how 
many students attended each lecture” (Clark 49). It also interesting that 
the professoriate resisted: “To this order the university wrote in protest . . . 
that such a request was completely unheard of” (49). Earlier, the Emperor 
sent spies to check up on the universities of the Holy Roman Empire, and 
professors had to give annual reports ( Professorenzetteln ) to the state on what 
they had taught that year (cf. Clark 50). 

See an interview with Professor Reiser, “Universities Are Being Turned 
into Schools.” It is not just disgruntled professors who have raised doubts 
about Bologna. As the process has unfolded, even some of the original 
framers of the program are exhibiting serious reservations. Jo Ritzen, 
president of the University of Maastricht (who helped to lay the founda- 
tion for the Bologna Process during his tenure as the Netherlands’ minis- 
ter of education), has recently admitted, “Bologna . . . has not brought 
what we hoped for and expected” (qtd. in Labi). This, after more than ten 
years of the process’ existence. 

1 ^ Nida-Riimelin is also the editor (with Werner Wcidcnfeld) of the influen- 
tial anthology on European identity, Europaeische Identitaet: Voraussetzungen 
und Strategien. For an interview with Nida-Riimelin, see “The Bologna 
Process Threatens to Founder.” 

] O 

° “Nida-Riimelin makes the point that, “... it must be said quite plainly that 
Bologna hasn’t facilitated their [students’] mobility but rather considerably 
restricted it” (“Bologna Process”). 

1 ^ Local attempts have been made to replicate Bologna in other parts of the 
globe as well, as witness the “Melbourne Model” introduced by the cost- 
cutting Vice Chancellor, Glyn Davis, who makes clear that his model, 
despite its local cognomen, is really a version of the Bologna model. The 
reforms created a storm of protest from professors and students alike. 
During the resistance, a leaked document revealed that the university 
administration had spent over 27 million dollars on marketing the model by 
2010 (“Melbourne Model”). 

- u For an authoritative — and horrifying — account what happened during the 
Bush Administration, see Diane Ravitch’s The Death and Life of the Great 
American School System: How Testing and Choice are Undermining Education. 

^ 1 See the University of Massachusetts at Amherst Office of Academic 
Planning & Assessment’s document, Program-Based Review and Assessment: 
Tools and Techniques for Program Improvement. 

" Cf.Jaschik, “Assessing Assessment.” Full disclosure: the author of the 
present essay attended the 2008 Teagle Foundation conference in North 
Carolina where this initiative was first discussed. 

no 

Zf3 Adclman argues that the United States should introduce reforms based on 
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the Bologna Process because it “is an analogue to the macroeconomic the- 
ory of convergence” ( Bologna Process ix). I find this rationale breathtaking. 
The equation of universities and economics would be troubling at any 
time, but given the utter failure of economics — especially macroeconom- 
ics — to predict or deal with the 2008 global financial meltdown, to cite 
such a discipline as a model borders on the irresponsible. 

A move recently urged for US colleges by Stephen Joel Trachtenberg and 
Gerald Kauver in the New York Times op-ed, “A Degree in Three.” 

This point was eloquently made in Bill Readings’ The University in Ruins , a 
book that rapidly is assuming the status of a classic. 

I use the term advisedly, not merely to express an outsider’s prejudice. See 
what the distinguished sociologist and student of management Burkard 
Sievers (professor at the Schumpeter Institut, Bergische Universitat 
Wuppertal) has to say in his deep analysis of how German universities are 
now set up, “The Psychotic University.” 

See the text of HB 2504 at <http:/ /www.legis. state. tx.us/tlodocs/ 
81R/billtext/html/HB02504F.htm>. 

OO 

40 Dickeson is author of a book called Prioritizing Academic Programs and 

Services. The Powell’s Books description of this text makes clear what the 
author means by prioritizing: the book “guides academic leaders through 
the process of ranking programs according to such critical factors as 
enrollment size and relevance to institutional mission. The book also 
includes successful strategies for suspending programs that hover on the 
margins of productivity and affordability.” Gf. <http://www.powclls.com/ 
biblio?isbn=9780787948160>. 

oq 

David Mazella, a professor at the University of Houston, contributes an 
essay to the present anthology that shows admirable circumspection in the 
very heart of Texas. 

on J ... . 

JU The University is, like all for-profits, run by a GEO, in this case a man 
named Stephen Tober who was appointed in June 2009, when his qualifi- 
cations were listed as: “nearly 20 years of experience in operations and 
management in a variety of industries including education, information 
technology services, investment banking and management consulting, 
most recently holding a senior position with ThinkEquity Partners in 
Chicago. Tober has also held senior roles in publicly traded companies 
including SPR, Leapnet and Smith Barney” (American InterContinental 
University). 

^ 1 For profits need such accreditation in order to be eligible for federal finan- 
cial aid, so that their students will have the possibility of transferring cred- 
its to other schools — and so they are eligible for the student loans that are 
a major source of income for such schools. 
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THE PYGMIES IN THE CAGE: THE FUNCTION OF THE 
SUBLIME IN LONGINUS 

W. ROBERT CONNOR 


In memory of Robert F. Goheen 

THE EDITORS OF THIS VOLUME have posed powerful questions, ones 
that go to the heart of the experience of reading and teaching literature. Are 
those experiences so “sublime” that they are beyond systematic analysis? Are 
they “ ineffable ?” 

The author of the ancient treatise On the Sublime took a strong stance on 
such questions, stronger, perhaps, than has been recognized. In what purports 
at first to be a handbook on rhetoric, Longinus gradually makes the case that 
the literary experience of the sublime has profoundly enriching and liberating 
effects on the individual. Without such experience, the soul shrinks and with- 
ers, never attaining its true greatness. The sublime, moreover, is not “ineffa- 
ble”; it can be recognized and analyzed, and perhaps given expression in new 
literature, using the techniques illustrated in the treatise. 

Thus, below the surface of this ostensibly circumscribed treatise is an implicit 
educational theory that speaks both to Imperial Rome and to our own age. 

Freak Show and Genre 

A few pages before the manuscripts of On the Sublime , ascribed to one 
Longinus, break off, there is a haunting image drawn from contemporary 
Roman society — its circuses, its freak shows, its taste for the exotic and the 
bizarre, its appetite for entertainment no matter what the cost in human 
degradation and suffering. The reader suddenly stands before a cage in which 
are locked small human beings from Africa: 

And so ... if what I hear is true . . . not only do the cages in 
which they keep the pygmies or dwarfs, as they are called, stunt 
the growth of the prisoners, but their bodies even shrink in close 
confinement. (44.5, Fyfe’s translation, modified)' 

The savagery of the practice is even more shocking in the Greek, for the word 
translated as “close confinement” is literally a “sack for holding the tongues,” 
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and the “tongues” are the thin reeds that make the overpowering sound that 
came from the ancient relative of the oboe, the aiilos. The pygmies, in other 
words, are confined in a tiny container, where their natural vocal capacity, as 
well as room to stretch and grow, is denied them. Although later the author 
refers to this place of captivity as a prison [desmo tenon), the image he prefers is 
that of this small pouch (glossokomeion) and the voiceless “tongues” within. The 
enslaved and confined pygmies have, in effect, been reduced to mute instru- 
ments of amusement for an entertainment-hungry populace. The image of 
such confinement leads to a powerful critique of slavery: “on the same princi- 
ple all slavery, however equitable it may be, might be described as a cage for 
the human soul, a common prison house” (44.5). This is a “common prison,” 
that is, one in which all are confined, the captives hauled to Rome from 
remotest Africa, their keepers, and all who are drawn into the spectacle. 

To be sure, Longinus does not claim as his own the powerful image of the 
captives in their confinement; indeed it comes, he says, from an unnamed 
philosopher whose views he does not find convincing (44.1). The image, how- 
ever, draws on ideas of enslavement and liberation already developed in the 
treatise and leads to Longinus’ own explanation of the absence of truly sub- 
lime talent in his day. But what, one wonders, does this image and the debate 
within which it occurs have to do with the literary quality of the sublime ( hyp - 
sos ) to which the work is devoted? As we look more closely at the literary form 
of the work, these features become all the more surprising. What, we ask, are 
the mistreated pygmies doing in a treatise of this sort? To answer this ques- 
tion, we will have to look more closely at the literary form of Longinus’ work. 

Although the work is filled with astute observations and close readings of 
literary texts, it cannot usefully be approached as “literary criticism” in the 
modern sense of that term. Its primary aim is not literary analysis or private 
delectation, but something “useful” ( chresimon ) for people in civic life ( andrasi 
politikois) (1.2). No wonder, then, that Longinus pays special attention to oratory, 
the literary form of greatest interest to those involved in public life. (Oratory in 
both Greek and Roman antiquity was regarded as a major literary form, whose 
techniques helped shape both poetry and other types of prose literature.) No 
wonder then that the treatise often cites and explicates the canonical orators, 
not least, the orator, Demosthenes. Other passages, including those from 
Homer — and other poets — are examined for what they can teach the aspiring 
orator. In chapter 15, for example, he draws on the great poets to illustrate the 
power that comes from the skillful use of images. 

A better approach to the treatise, though still imperfect, is to recognize it 
as an extension of traditional rhetorical training manuals, the so-called “arts” 
of rhetoric. (“Art” in this sense is simply a reflection of the translation of 
Greek techne into Latin ars. The Rhetorica ad Alexandrian [“Rhetoric for 
Alexander”] preserved among the works of Aristotle, is a good example of 
such technai and is still one of the best introductions to ancient rhetoric.) 

At the outset of the treatise, Longinus indicates he is writing within the 
tradition of technologiai, that is, treatises about a craft or skill (1.1 ). ~ The 
work thus carries on the tradition of rhetorical how-to-do-it treatises that 
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began in the fifth century before our era, with the shadowy Corax and 
Tisias, and those that, thanks to the criticisms of Plato, are far better 
known — teachers of rhetoric such as Protagoras, Hippias, Gorgias, and 
other “Sophists.” (As is well known, the word sophists in ancient Greek need 
not imply deception; it simply denotes a professional teacher of rhetoric, 
whose services were much in demand in ancient Athens and other demo- 
cratic citiesA) The tradition continued through the Hellenistic, Roman, and 
Byzantine periods. It was still flourishing in the tenth century of our era 
when the earliest and best of the surviving manuscripts of On the Sublime, 
Paris 2036, was copied. 

Longinus, it is clear, is well aware of this tradition. He may even have 
written such a manual himself. 4 The first few pages of On the Sublime, more- 
over, speak in practical, utilitarian terms, and identify the treatise as a technolo- 
gic. Only gradually does it move beyond the limits of such treatises. At the 
outset, it asks to be approached in part as a “technology of the sublime” (2.1 
f), that is, as a practical guide designed to show the aspiring orator or writer 
how to launch thunderbolts that will stun his audience into earnest submis- 
sion. This technology is built on the conventional foundation for oratorical 
achievement, the capacity for powerful speech (8.1). It uses, moreover, the spe- 
cialized vocabulary and analytical techniques of ancient rhetoric — for exam- 
ple, “amplification” ( auxesis ) in chapters 1 1 and 12. It may then be viewed as 
an effort to build on this tradition by developing the idea of “the sublime” 
already brought into circulation by Longinus' predecessor, Caecilius of 
Calacte (1.1). 

By casting his discussion of the sublime in the mold of a rhetorical trea- 
tise, Longinus does two things. First, he establishes the tension or paradox that 
underlies the whole treatise — the idea that something that appears transcen- 
dent can be realized through mundane means. Second, he emphasizes that the 
importance of the sublime is in the first instance rhetorical; it helps the orator 
achieve a moment that “scatter [s] everything before it like a bolt of lightning 
and reveal[s] the full power of the speaker at a single stroke” (2.4). For this 
purpose, the treatise provides examples both of passages that achieve this 
effect and of those that attempt it but fall short. 

The Sublime as Philosophy 

The writing of a rhetorical techne was still a fully respectable undertaking 
in the Roman Empire when, whatever its exact date, this treatise was com- 
posed. At the end of the day, anyone who could expand this venerable tradi- 
tion with a richer vocabulary or new techniques could leave the study feeling 
pride and accomplishment. But in the middle of the night, doubts or cultural 
warfare might trouble the sleep. Rhetoric had long been subjected to powerful 
scrutiny, not least by Plato and his followers. His criticism of rhetoric, perhaps 
most powerfully articulated in the Gorgias, can be generalized to any technolo- 
gy. Since rhetoric, the speaker’s technology, can be used for either good or ill, 
it is not, in Plato’s view, an end in itself, but must be subordinated to moral 
principles. 
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Teachers and theorists of rhetoric have tried to find a satisfactory resolution 
to the tension between rhetoric and philosophy, or at a more general level 
between power and morality, beginning as early as Plato’s Republic and his con- 
temporary Isocrates, and continuing through Quintilian’s reaffirmation of a dic- 
tum of Cato the Elder, that the good orator must be a vir bonus dicendi peritus (qtd. 
in Quintilian 12.1), that is, someone both morally good and skilled in speaking. 

Where does Longinus stand in this contention between the rhetoricians 
and the philosophers? If the Platonic criticism had validity against older forms 
of rhetoric, it would apply a fortiori to the even more powerful technology, the 
art of the sublime, which Longinus was developing. We might expect him then 
to be defensive, or perhaps to turn his back on this debate as some writers of 
technai did, but not Longinus. Every now and then, philosophy breaks through 
the technological surface of the treatise. However, it is not the unregenerated 
philosophical critique of rhetoric propounded by Plato and Socrates, but the 
significantly modified philosophy of the Roman Empire. By the time this trea- 
tise was written, “philosophy” had come to mean in common parlance not 
logic, metaphysics, or cosmology, but detachment from conventional values of 
power, wealth, prestige and, not least, from the ravenous appetite for amuse- 
ment and entertainment. Although the various philosophical schools — 
Academics, Stoics, Epicureans, Neo-Platonists and so on — often differed with 
one another, they shared a wariness about the dominant cultural values of 
their time. “To disdain” or, literally, “to think down” (to kataphronein ), was an 
identifying mark of the philosopher, for “philosophy” had become essentially 
counter-cultural, a steady critique, often associated with bearded ascetics, of 
what we might call “the Roman way of life.” 

It is the philosopher’s voice that speaks in passages such as this: 

... it is a mark of greatness to look down upon ( kataphronein ) . . . 
wealth, status, reputation, unlimited power and everything else 
that has great external and theatrical appeal; to no sensible man 
would these appear to be high up on the scale of goodness, since 
it is no mean good to think one’s way around these things. (7.1, 
my translation) 

At the very outset of the work, Longinus has made an even more dramat- 
ic move away from conventional rhetoric. 

. . . the sublime consists in a consummate excellence and distinc- 
tion of language, and . . . this alone gave to the greatest poets and 
prose writers their pre-eminence and clothed them with immortal 
fame. For what surpasses natural ability leads listeners not to per- 
suasion (peitho ) but to ekstasis (1.3, Fyfe’s translation, modified). 

But what does he mean by this extraordinary assertion? What does he mean 
by ekstasis ? 
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Ecstasy and Enthusiasm 

The art of rhetoric, Longinus implies, puts us in control of things; the 
sublime, on the other hand, cannot be resisted: 

Everything that startles us and makes us marvel constantly forces 
us beyond the persuasive and the enjoyable. The persuasive 
indeed is for the most part within our control, but these things 
establish a tyrannical hold and irresistible force over every listen- 
er. (1.4, my translation) 

From ekstasis , we derive English “ecstasy,” a drug, a kind of euphoria, a hyper- 
bole for pleasure and excitement. For the ancients, the word pointed in quite 
different directions. While the word has technical uses in medicine and other 
fields, its etymological sense, “standing away from something,” often comes to 
the fore in discussions of extreme varieties of physical or religious experience. 
In such cases, it was commonly thought that a person’s own breath or spirit 
( pneuma ) had gone out from him or her. The next stage might be that some 
other spirit entered in. This state is now commonly referred to as “possession,” 
that is, a state in which a person feels that he or she has been taken over by a 
divinity who may use that person’s body to speak or act. The entrance of the 
divinity might also be called by a counterpart term, enthousiasmos, from which 
the English “enthusiasm” is a pale derivative. The Greek term is a combina- 
tion of en, in, and theos, god. 

The entrance of the divinity might involve spasms, frothing at the mouth, 
seizures similar to those in epilepsy, known in antiquity as the “sacred disease.” 
Sometimes the possessed person would deliver prophecies, in prose or even in 
verse. This is perhaps the best way of approaching the prophecies emanating 
from the prophetess at Delphi, and from the Cumaean Sibyl and other famous 
and frequently consulted oracles in antiquity. The most familiar modern anal- 
ogy may be the speaking in tongues by Pentecostals and others. 

Ekstasis and enthousiasmos in Longinus both allude to this pattern of reli- 
gious possession. The terms are counterparts. Etymology shows the relation- 
ship between the two terms, the one beginning with the word for “out,” ek, the 
other with the word for “in,” en. Longinus uses this vocabulary to talk about 
an otherwise difficult-to-articulate effect of the sublime. It is still not easy. 
Longinus gropes for terms, stretches syntax to the extreme, forces one 
metaphor within another. We can understand some of what Longinus is trying 
to express because we know that literature can sometimes “take our breath 
away,” but that catches only part of what he has in mind. He recognizes the 
power of sublime passages to shock and awe an audience, but is no less inter- 
ested in the possession that accompanies such ekstasis. Here, for example, is 
what Longinus says about one of the techniques he recommends, the direct 
expression of emotion — it must be genuine emotion — at the proper moment 
in a speech or text. Some of his comments on the matter are a good illustra- 
tion of what we have just noted about the complexity of his style. The follow- 
ing translation is as literal as possible in order to catch the metaphors: 
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... as if, under the influence of some frenzy ( manias ) and breath 
(pneumatos), to let in the divinity as one lets out one’s breath (ent- 
housiastikos ekpneon), and, as it were, to be possessed by Phoebus 
[Apollo]. (8.4, my translation) 

The expression of intense emotion, in other words, brings listeners to a 
state analogous to possession in a religious cult. They experience something 
similar to what Pentecostals today might call the Holy Spirit. The metaphors 
in each case are not about some attenuated “spirituality,” but an intense physi- 
cality. Participants feel the breath of a divinity. It fills them as they let out their 
own breath and let in the divine. 

The language of religious possession recurs throughout the treatise. In 
chapter 10.3, for example, Longinus extrapolates from the language of a 
poem by Sappho that the poetess was inspired as if possessed by Apollo 
( phoibatai)P A little later, in discussing Demosthenes’ speech, On the Crown, 
Longinus uses a related word as he imagines the orator: “... as it were sudden- 
ly inspired by a divinity and, in effect, seized by Apollo phoiboleptos)” (16.2). 

But again three little syllables, “as it were” ( kathaper ), draw the line between 
what Longinus sees in Demosthenes and true religious frenzy. Longinus is not 
suggesting that the effect of sublime literature is to create a moment of reli- 
gious ecstasy in either speaker or audience. Rather, metaphors drawn from 
religious possession provide him with a discourse that was not available in the 
traditional technical vocabulary of rhetorical instruction. 

Soul Talk 

Vocabulary drawn from ancient religion provides an instrument of great 
value to Longinus in his discussion of the ability of well-craftcd language to 
astonish and overpower an audience. But religious-like terminology in On the 
Sublime is not limited to this purpose. In the form of “soul talk,” it reaches 
more broadly and more deeply. It allows Longinus to begin an exploration of 
the effects of sublime language in the deepest parts of our being. 

Note, for example, how in discussing the arrangement of words, Longinus 
extends an analogy between oratory and melody in music. He alludes at first 
to an instrument widely thought to induce frenzy, the aulos (the reed instru- 
ment, similar to the oboe, mentioned at the beginning of this essay): “Does 
not the aulos ... induce certain emotions in those who hear it? Does it not 
seem to carry them away and fill them with divine frenzy?” (39.2). Longinus 
then uses the example of the aulos to suggest that skillfully arranged language 
in a speech — “which is a kind of melody in words” — can “take hold not of 
the listener’s ears alone but of his very soul” (39.3). The effect reaches far 
beyond persuasion. It is like a spell cast upon us. But this is white magic, not 
black, for it turns us towards what is “weighty, deserving of respect, and sub- 
lime” (39.3). 

Longinus’ “soul talk” often seems to embarrass his translators. They 
squirm to find other translations for the word psyche and its relatives — “heart,” 
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“our thoughts,” or simply “us” (8.4, my translation). Longinus, however, has 
few hesitations about using the term. In fact, the little qualifying words that 
mark most of his religious terminology as metaphor or analogy are usually 
absent when he speaks of the soul. For him, the soul is not a metaphor but a 
reality without which the body is limp and empty (1 1.2). It may be the locus of 
the emotions but it is also profoundly affected by them. Indeed an emotion 
may be defined as a “movement and upheaval of the soul” (20.2). Thus, it can 
be deeply affected by the sublime: 

For by its very nature our soul is somehow lifted up ( epairetai ) by 
the truly sublime, and having acquired a certain pride and lofti- 
ness, is filled with joy and great self confidence ( megalauchia ), as if 
the soul itself had created what it heard. (7.2, my translation) 

This ability of the sublime to lift up the soul can even bring mortals close to 
the divine, as Longinus indicates in chapter 36.1, “The sublime raises one up 
close to the lofty mindedness ( megalophrosune ) of God” (my translation). 

Here, as so often, Longinus’ discussion of the soul turns on a central axis 
in the work, a polarity at one end of which is greatness or height and at the 
other meanness and lowness. Since in Greek, words compounded from mega 
can convey height as well as mass, “soul talk” is height talk. These words are 
part of a metaphorical nexus that ties together images of elevation, verbs of 
lifting up, and, of course, the sublime itself; hence the translation “lofty mind- 
edness” just used for megalophrosyne. This nexus can easily be obscured by 
translations that speak blandly of “greatness” or “great writing” when the 
treatise is concerned with ideas of reaching upward, tallness, loftiness. The 
sublime (to hupsos) is, after all, a word for height, not mass. 

Longinus’ thinking about the soul becomes clear when the vocabulary and 
imagery of height are kept in view. In discussing Euripides, for example, 
Longinus concedes at the outset that the tragic poet was not megalophues (5.3) — 
he lacked a natural inclination toward the grand and lofty — but rather, sug- 
gests that Euripides successfully forced himself toward such qualities. Longinus 
then supports this view by citing several passages, including one from the lost 
play, The Phaethon. In this play, Helios allows his son to drive the chariot of the 
sun — with catastrophic results. Longinus asserts that the soul of Euripides 
shared the perilous flight of those winged horses. “It could never have imag- 
ined such things if it had not run neck and neck with those heavenly doings” 
(15.4). The emphasis on the soul in this passage is doubly interesting. Longinus 
clearly thinks that it is the soul that makes it possible to imagine the situation. 
In addition, the image of the soul reaching up to heaven is not a new thought 
injected at the end of the analysis of Euripides, but is implicit in the “great- 
ness or loftiness” language of the term megalophues , used at the very outset of 
the discussion. 

An Educational Theory 

The “soul talk” in the treatise points toward an educational theory. To be 
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sure, the full scope of the theory is left implicit for the most part, and is largely 
eclipsed by the concentration on rhetoric. Rhetorical training was a central 
part of Greek and Roman education for two millennia, from Classical Athens 
to the fall of the Byzantine Empire. 

Indeed in late antiquity and the early Byzantine period, Greek education by 
and large avoided the relegation common in Western Europe, of rhetoric to one 
of three preliminary educational roads (the trivium, consisting of grammar, rhet- 
oric, and logic). It was rather one of the most prominent of a range of skills 
( technai ) widely regarded as important for the society. In 425, for example, when 
Theodosius established in Constantinople a pandidakterion, the first “university” in 
Europe, there were thirty-one chairs: ten each in Greek and Latin grammar, two 
in law, one in philosophy, and eight in rhetoric.® There were, apparently, no 
chairs in medicine or theology, nor in what in the West became the quadrivium, 
that is, arithmetic, geometry, astronomy, and harmony. The quadrivium and trivi- 
um, as their Latin names suggest, were developed in the West, probably in the 
fifth century of our era, flimsy, one might suspect, against the flood of bar- 
barism, which rose more swiftly and more disastrously in the West than in the 
East. In the Greek-speaking East, after grammar, rhetoric ruled supreme. 

While Greek culture was always deeply concerned with paideia, that is, 
education in all its aspects, very few works dedicated to that subject were com- 
posed. The art of rhetoric, by contrast, was the subject of many treatises and 
discussions. Against this background, a few comments in On the Sublime stand 
out in high relief. While Longinus shares the adulation of rhetoric so wide- 
spread among his fellow Greek speakers, he hints at the outset of his work that 
he intends to address the topic of education more broadly. Longinus criticizes 
his predecessor, Gaecilius of Galacte, for failing to “deal with the means by 
which we might be empowered to lead our natural capacities toward some sig- 
nificant progress in greatness and loftiness” (eis posen megethous epidosin ) (1.1, my 
translation). He drops the topic for the time being, perhaps because he needs 
first to clarify what he means by “greatness and loftiness,” which, as we have 
seen, is closely linked to the sublime. That clarification emerges in part 
through his discussion of how Euripides forced ( prosenagkasen ) his nature “into 
the tragic mould” (15.3). Borrowing a term often used for heroes and others 
who undergo great labors, Longinus calls Euripides philoponotatos, “the greatest 
lover of toil” (15.3). 

But hard work cannot be the whole basis of education. Nor is it simply 
exposure to, or analysis of, the passages he considers sublime. That is too pas- 
sive and too easy. Plato, he notes, points in a different direction — to an active 
process, and a competitive one as well, involving two crucial components — 
imitation ( mimesis ) and rivalry ( zelosis ): “What is this road and what is it like? It 
is imitation and rivalry of the great prose writers and poets of the past” (13.2, 
my translation). Mimesis is frequently used in ancient discussions of literature 
to describe the relationship between writer and subject matter or writer and a 
tradition on which he or she draws. It is clear that for the Greeks, such imita- 
tion was not an attempt at exact replication, but adaptation — creative restate- 
ment. But what of zelosis, the word from which English derives both “zeal” 


104 



THE PYGMIES IN THE CAGE: THE FUNCTION OF THE SUBLIME IN LONGINUS 


and (in all probability) “jealousy”? The translation “rivalry” catches only part 
of the meaning since the Greek term reflects the motivation behind mimesis — 
driving ambition, competition, and sometimes envy as well. 

The importance of zelosis in Longinus’ thought is evident in a difficult, but 
crucial passage, another analogy drawn from Greek religion. After relating the 
familiar story that the prophetess of Apollo at Delphi was inspired by breathing 
vapors coming from a cleft in the bedrock, Longinus draws an analogy to the 
inspiration (in the literal sense of breathing in) that comes from great writers: 

Thus, from the grand and lofty nature of those of long ago a 
kind of outpouring is brought into the souls of those who aspire 
to rival them ( zelounton ), as if from the mouths of the sacred. 

(13.2, my translation) 

What does it mean to be a “rival” of one of the great writers of the past? 
Longinus means — as do many ancient authors who use these terms — surpass- 
ing the classics, outdoing models of acknowledged eloquence and wisdom. 
Longinus’ purpose, we see once again, is not to write “literary criticism” but to 
help others — his contemporaries, his students perhaps — to write and speak at 
the highest possible level. In today’s world, he might feel more comfortable in 
a creative writing or perhaps even communications program, rather than in 
English, comparative literature, or classics. He would not, if I read him cor- 
rectly, be satisfied if a student wrote a fine critical essay; he would want to 
hear words that thunder louder than Demosthenes, or to read a passage even 
grander than Homer. 

So the student, in this view of education, cannot be a passive admirer of 
past greatness; he or she must compete with it and try to surpass it. This becomes 
clear in one of the most powerful — or sublime — passages in the work: the 
analogy it draws between human life and the great Greek national festivals 
( panegyreis ) of which the Olympic games are the best known example. It is part 
of our very nature, Longinus asserts, to stride forward into competition, just as 
contestants do in festival games: 

For nature (physis ) has not decided that human beings are lowly or 
ignoble creatures, but has led us into life and into the whole uni- 
verse as if we were going to be spectators ( theatai] of all of it in 
some great festival — or rather contestants ( agonistai ) eager for 
honor. Nature has lost no time in inspiring ( enephusen ) our souls 
with an irresistible passion ( erota ) for whatever is great and, by 
comparison to ourselves, more divine. (35.2, my translation) 

In this passage, in fine rhetorical fashion, Longinus corrects himself, saying we 
are not onlookers but contenders, competitors in the games, seeking to be 
honored. That impulse is built into us. It is part of our souls. Implicitly it 
makes us admirers and rivals of the achievements of other contenders, past 
and present. 
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Longinus is not alone in drawing his imagery, and perhaps his theory, 
from athletics. Theon of Alexandria in the first century of our era developed 
rhetorical instruction based on progymnasmata — exercises requiring the pupil to 
produce speeches of various types on subjects drawn from history and mythol- 
ogy Later Aphthonios of Antioch developed a textbook with exercises in four- 
teen different categories such as encomium, fable, maxim, confirmation, refu- 
tation, ekphrasis , and invective. 7 The athletic imagery in Longinus parallels that 
of progymnasmata and evokes further questions. What happens, after all, to the 
contestant in such a setting? The body is stretched and strengthened; it grows 
stronger, taller, more muscular, and more vigorous. By analogy, we might 
expect, the soul when it imitates, and then surpasses its models, should be ever 
more vigorous and reach ever higher. Ultimately, this kind of athlete becomes 
a person of true loftiness of spirit ( megalopsychos ). 

One can readily imagine the treatise concluding not long after the image 
of the competitor striding into one of the great pan-Hellenic festivals. One 
part at least of a powerful, ambitious, educational theory is now before us. 

The student must not get away with recognizing, analyzing, praising great 
writing. Even mimesis , creative adaptation, by itself is not sufficient. One must 
surpass the past. 

To stop at this point, however, would be to avoid difficult questions. 

What prevents such zelosis ? Why is it so rare? The answer to that question 
requires us to look at the other end of the axis around which the treatise 
turns — the downward tendency toward corruption and the wasting away of 
the soul. 

The Pygmies 

In its final pages, having moved from the modest genre of rhetorical 
handbooks to a powerful and evocative image of the pan-Hellenic festival, the 
treatise metamorphoses again, this time into a dialogue or debate. An inter- 
locutor steps in, an unnamed philosophical friend of the author. Although 
such antilogies are a well-attested feature of ancient debates, they are not the 
way rhetorical handbooks went about presenting their material. They belong, 
in other words, to a different genre. And in On the Sublime, as the genre changes, 
so does the thrust of the argument. Longinus now confronts a problem in the 
theory behind the first part of the treatise. How are we to explain the absence 
of truly sublime literature in the present age? 

We find natures that are supremely persuasive . . . and especially 
rich in literary charm, yet really sublime ( hupselai ) and transcen- 
dent ( hypermegetheis ) natures are no longer, or only very rarely, now 
produced. (44.1) 

The observation restates the question raised by our analysis of Longinus’ edu- 
cational theory: what keeps us from zelosis ? Both Longinus and his philosophi- 
cal interlocutor, who now enters the discussion, agree on the point that the age 
in which they live is deficient in the truly sublime. Both use metaphors of 
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enslavement. But they have very different views on how the sorry state of con- 
temporary culture is to be explained. The interlocutor adduces his explanation 
with some diffidence. He knows, surely, that since the Roman Empire does not 
allow true democratic freedom, his theory will seem out of date and irrele- 
vant, but he propounds nonetheless “the hackneyed view that democracy is 
the kindly nurse of genius” (44.2). He suggests that only political freedom has 
the ability to nurture the thoughts of great intellects. But now “we seem to be 
schooled in an equitable slavery” (44.3) and do not even taste the true source 
of great oratory — freedom. It is here that the interlocutor introduces the 
image to which the title of this essay alludes: 

“And so,” my friend added, “if what I hear is true that not only 
do the cages in which they keep the pygmies or dwarfs, as they 
are called, stunt the growth of their prisoners, but their bodies 
even shrink in close confinement, on the same principle all slav- 
ery, however equitable it may be, might well be deemed as a cage 
for the human soul, a common prison.” (44.5) 

The anonymous philosopher’s conclusion is clear — all slavery imprisons our 
souls. No wonder then, since all are entrapped in the benign autocracy of the 
Roman Empire, that no truly sublime literature results. 

The image of the stunted pygmies is perfect for the thematics of the trea- 
tise. We have glimpsed grandeur and loftiness of soul, now we see its opposite. 
Yet the image is presented through the voice of the interlocutor, not of the 
author himself. The author distances himself from this socio-political explana- 
tion in favor of a highly moralistic one. He is willing to accept the idea that in 
this world everyone is enslaved, and even that such enslavement prevents sub- 
lime literary expression. But he reverses the interlocutor’s critique of the loss 
of freedom under the Pax Romana , pointing instead to a perpetual but inward 
warfare: 


But consider. Perhaps it is not the world’s peace that corrupts great 
natures but much rather this endless warfare which besets our 
desires ( epithurnias ), yes, and the passions that garrison our lives in 
these days, and make utter havoc of them. It is the love of money, 
that insatiable sickness from which we all now suffer, and the love 
of pleasure that enslave us, or rather, one might say, sink our lives, 
“with crew and all” into the depths ( katabuthizousin ). (44.6) 

The image of the sinking ship replaces that of the caged pygmies, and drives 
home Longinus’ own critique. His well-heeled, well-educated readers — not 
some captives from a far off land — are the true slaves and the ones who are 
pressed downward. And we are ourselves (for Longinus uses the first person, 
not the third) the cause of our misery since we have chosen a life of pleasure 
and greed. Not content to rely on the image of the shipwreck, Longinus 
launches a whole fleet of new metaphors. Greed is a sickness, and not just any 
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sickness but one that makes its victims shrink and wither ( noserna mikropoion); we 
waste away, malnourished, because of it. Greed inverts the proper relation of 
man and god. No longer does a divinity come inside us and inspire us; rather 
we go out and make gods ( ektheiasantas ) of the evil that descend from wealth. 
These evils, among them personified Extravagance, march into our cities and 
our homes, once Wealth has opened the gates for them. They nest there, like 
noxious birds, and breed their nestlings, Swagger ( alazoneia ), and Delusion 
( tuphos ) and Luxury (truphe). These in turn, if they reach maturity, breed 
Hybris, Transgression (paranomia ), and Shamelessness ( anaischyntia ), the inex- 
orable masters of our enslaved souls (44.7). 

And what is the end of this? Greatness of soul withers and starves to 
death (phthinein de kai kata?narainesthai ta psychika megethe) (44.8). Under such cir- 
cumstances the soul can not reach to the sublime nor can greatness flourish. 

Is, then, the experience of the sublime and of its results, greatness of soul 
and sublime writing, impossible in an age enslaved by wealth and pleasure? 
Longinus holds out little hope: “This must inevitably ... happen ...” (44.8); “... 
the whole life of each one of us is now governed wholly by bribery ... we have 
sold our souls for profit ...” (44.9). Amid all this — if we still possess any 
grandeur of soul — it loses its zeal ( azela ginesthai) (44.8). And although the casti- 
gation becomes slightly less universal, applying to “all but a few of us” in 
44. 1 1 , there is no hint that studying great literature, or experiencing the sub- 
lime, will free us from our slavery to wealth and pleasure. Indeed the relation- 
ship between freedom and education seems just the opposite: we must first free 
ourselves from addiction to wealth and pleasure for then , and only then, is 
there some chance that mimesis and zelosis will do their work and we might 
achieve, even in the present dismal age, some grandeur and loftiness of soul. 

Is even that too optimistic? Are we, ancient Romans and contemporary 
Americans alike, hopelessly enslaved by “the emotions (ta pathe) that garrison 
our lives?” (44.6). Longinus’ answer depends on his understanding of the emo- 
tions, and perhaps, of the effect that the experience of the sublime has on the 
audience, even in adverse cultural circumstances. When the sublime induces 
ekstasis, for example, are we temporarily freed from emotions that might other- 
wise drag us to lower levels of experience, expression and action? Gan the 
Socratic cross-examination, the Platonic dialectic liberate us from base emo- 
tions if used consistently and forcefully enough early in an education? My sus- 
picion is that the treatise went on to discuss the emotions along the lines sug- 
gested, but after promising a discussion of the emotions, the manuscripts end 
in mid-sentence. 

Longinus on Trial 

Much remains to be resolved when the manuscripts of On the Sublime 
break off, but clearly its author has answered at least some of the questions the 
editors of this collection have put to him. In his view, even the most sublime 
passages of literature are capable of identification, explication, and evaluation. 
Longinus not only implies, he demonstrates, that there is nothing “ineffable” 
about the sublime. 
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But when viewed as an educational theory, his treatise is polemical, deeply 
counter-cultural and likely to provoke anger, then and now. It is easy to imag- 
ine his critics linking arms to demand Longinus’ condemnation. One might 
imagine the proceedings to be of the following sort: 


Prosecutor: Your honor, I charge the defendant, Longinus, author 
of the scurrilous tract On the Sublime , with fraud, and the corruption of 
the young. He has misrepresented his work as a handbook of rhetori- 
cal instruction when it is in fact an attack on our culture and its most 
deeply held values. It turns out to be a manifesto that will mislead all 
who read it, not least our young people. 

Judge: How plead you, Longinus? 

Longinus: Proudly guilty as charged, your honor. I contrived a trea- 
tise that looked like a rhetorical handbook and transformed it into 
something much more interesting and important. If this is fraud or 
misrepresentation, I am guilty. If its effect is to displace an education 
based on self-gratification, and avarice, then I will happily pay the 
penalty. 

Judge: And, Mr. Prosecutor, what penalty do you seek? 

Prosecutor: Exile, your honor. Exclusion from the company of all 
who teach and all who learn. 

Judge: The court will need to determine whether there was intent to 
deceive. 

Longinus: Objection, your honor. Deception has not been estab- 
lished. 

Prosecutor: Longinus has himself admitted that he “transformed” 
his treatise into something quite different. It turns into a diatribe 
against the existing social order. Only malice aforethought can 
account for such a vicious attack on our society. 

Longinus: I transformed the literary form of my writing because my 
purpose is transformation — the transformation of those who read my 
work. 

Prosecutor: Transformation? I submit, your honor, that the only 
transformation he had in view was the transformation of his bank 
balance. But, as your honor has no doubt noticed, the defendant has 
admitted his deception, and its intended effects on his readers. I 
therefore, ask for a directed verdict of guilty. 

Judge: Lest anyone accuse us of being prejudiced or peremptory, the 
court will overlook the defendant’s admission and enter a plea of “not 
guilty” on his behalf. The prosecution may present its case. 
Prosecutor: The case is quite simple, your honor, but not as simple 
as the defendant’s claims in this absurd pamphlet. In order to sell 
copies of it, the defendant has made repeated fraudulent claims about 
the study of literature. He contends that the purpose of such study is 
to encounter the sublime or the ineffable, when everyone knows the 
purpose of literature is entertainment. 
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Longinus: I admit that I believe that literature has a higher purpose 
than entertainment. I also admit that in my writings I have used 
words to speak about words. I notice, however, that the prosecution 
also uses words to show what may or may not properly be said about 
literature. Clearly then literature is not ineffable, nor incapable of 
being analyzed. 

Prosecutor: Objection your honor. Double negatives should be 
banned from the courtroom as unnecessarily confusing. 

Longinus: A mere litotes, your honor. 

Judge: Objection sustained. The defendant will avoid rhetorical fig- 
ures and confine himself to language simple enough to be understood 
by the prosecution. 

Longinus: Then let me state that I have never said the sublime was 
“ineffable.” In fact, I think such language deserves no place in the 
study of literature. In fact, since the term applies properly only to 
God and his works, your honor, may I bring a charge of blasphemy 
against the prosecutor? 

Judge: The court does not recognize such an accusation. Return to 
the case at hand. 

Prosecutor: I will allow the matter of the sublime and the ineffable 
to pass. No one cares in any event about such pedantic distinctions. 
The point is not whether sublime passages can be identified, explicat- 
ed and analyzed using the flimsy and implausible techniques promul- 
gated in this pamphlet. That is not the issue. 

Judge: Perhaps the prosecution will explain to us, then, what is the 
issue. But may I remind you, counselor, that the prosecution must 
show that someone has been or will be harmed by the defendant’s 
conduct. 

Prosecutor: Of course, your honor. The defendant’s conduct has 
the effect of wasting time and energy on the study of literature. 

Follow Longinus’ course and you — that is our society, not you person- 
ally your honor — will end up displacing a truly practical education, 
one based on the three F’s: facts, figures and formulae. 

Judge: If I follow you correctly, counselor, all studies that are not 
based on these three F’s should be prohibited — not just Longinus’ 
approach to rhetoric but philosophy, history, music, and the other 
arts. Would that include the study of law? 

Prosecutor: With all due respect, your honor, I had hoped that it 
would be clear that my claim is much narrower. I only claim that 
Longinus’ approach to literature and to education puts the student on 
a slippery slope. A few innocent sounding observations about Homer, 
Demosthenes or some other deservedly obscure author lead to dilet- 
tantism, and the waste of time and resources. These are drawn away 
from other more practical instruction and from the profit motive that 
has made our society great and prosperous. Those who follow 
Longinus will end up unemployed, alienated, and impoverished, with 
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heads in the clouds and purses empty. 

Longinus: Bravo! Well spoken! You show a fine capacity for lofty 
oratory Mr. Prosecutor. I am honored to be accused by you. 

Judge: Silence. I warn you, Mr. Longinus, against further interrup- 
tions. Do you wish to call any witnesses, counselor? 

Prosecutor: None is needed, your honor. I myself provide the nec- 
essary proof of the fraudulence of the treatise. In it the accused con- 
tends that by contemplating and analyzing certain passages one can 
experience the sublime and thereby become a more effective orator or 
writer. Yet in my reading of widely appreciated and highly successful 
authors of the present day I have never encountered passages that 
could conceivably be called “sublime,” nor has such reading ever pro- 
vided me with any practical benefit apart from entertainment. Many 
distinguished literary critics have written to confirm my judgment. On 
this basis, the prosecution rests its case. 

Judge: Your argument cuts to the heart of the matter, counselor. I 
am impressed by how little you have learned from the study of litera- 
ture. 

Prosecutor: Many others have studied more intensely and gained 
even less than I have. 

Judge: I congratulate you for your modesty, counselor. But let us see 
now what response the defendant will make, unless, of course, he is 
dumbfounded by the sublimity of the prosecution’s argument. 
Longinus: On the contrary, I am always inspired by true eloquence. 
You see, I believe, that sublime literature can bring our souls very 
close to the grandeur of God. 

Prosecutor: “Souls!” Your honor, you have heard it from his own 
mouth — the defendant has violated yet another legal principle — the 
constitutional separation of soul talk and literature. By asking one fur- 
ther question I will show how seriously this aggravates the defendant’s 
misconduct. 

Judge: Proceed. 

Prosecutor: Do you have a valid license to use the word “soul”? I 
point out to the court that all such licenses expired two hundred years 
ago and have not been renewed. 

Longinus: Guilty again. I have no such license, nor would I ever 
apply for one. But in turn I ask you, distinguished prosecutor, one 
question: can you show that two hundred years without “soul talk” 
have enriched the understanding of literature, or our lives? 
Prosecutor: It has freed us from all sorts of error, notably religious 
dogma and what you call inspiration and enthusiasm. 

Longinus: And in doing so it has, I submit, depleted the understand- 
ing of why literature should be studied. This impoverishment of lan- 
guage has debased the education of our young people. It locks both 
teacher and student into low and narrow linguistic cages. Such con- 
finement stunts intellectual and personal growth. It is criminal to 
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defraud people by calling such impoverishment “education.” 

Judge: I must remind the defendant that it is he who is on trial here, 
not the prosecutor. You may respond to the defendant’s comments if 
you choose, counselor, but you are under no obligation to do so. 
Prosecutor: Thank you, your honor. I will simply ask one further 
question, one that brings us to the core of the matter, and to our rea- 
son for bringing this case in the first place. You said, Mr. Longinus, 
that sublime literature can bring us close to the grandeur of God. Do 
you believe that one can measure how close one has thereby come to 
this “God” of yours? 

Longinus: By no means, but ... 

Prosecutor: Alia! You admit then that all this “soul talk” is vague 
and immeasurable. Head in the clouds again, eh, Mr. Longinus? 
Longinus: I have no way to measure closeness to the divinity, but I 
believe we know some of the steps that can bring a person closer to 
such experience and can determine whether those steps are being 
taken or not. 

Prosecutor: I presume that such steps include reading sublime liter- 
ature and talking about what makes it sublime. A waste of time, to be 
sure, but even worse, tell me this, how could one possibly know 
whether these steps lead anywhere? 

Longinus: Here is one way. You might ask how often there are sub- 
lime passages in your own work, or that of students who have studied 
with you. This will not let you measure your distance from the 
grandeur of God, but you will in that way have some indication of 
whether the experience of the sublime is bearing fruit. 

Prosecutor: We are talking about the study of literature, not cre- 
ative writing. 

Longinus: If those two are to be detached from one another, you 
might prefer I told you another way? 

Prosecutor: Another way? 

Longinus: Yes, the really important one. The reason we value the 
sublime is that it nurtures a loftiness and grandeur of soul — what we 
call megabpsychia. When the study of literature no longer had the soul 
in view, this term went out of circulation. But I am sure that you, 
learned counselor, understand what is meant by it. 

Prosecutor: Of course I do. It comprises generosity, courage, bold- 
ness and . . . 

Longinus: And? 

Prosecutor: And much, much more. 

Longinus: Look around you, then. Is generosity and courage and 
boldness and “much, much more” what you see? Are these the quali- 
ties you find among your colleagues? When you sit in the company of 
the most learned or the richest or the most powerful do you find mega- 
lopsychia, or pettiness, meanness, narrow mindedness, pedantry, greed? 
Which is it? The sublime, you see, nurtures grandeur of spirit and lifts 
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us up; it helps us free ourselves from enslavement to dclusionary 
pleasures and desires. You might say that sublime literature, properly 
studied, is an instrument for a truly liberating education. That is the 
kind of transformation I had in view in writing the treatise. 
Prosecutor: Your Honor, the defendant has several times admitted 
his guilt. We do not need to listen to his sermonizing. He has clearly 
violated the protocols of literary study and perpetrated a theory of 
education that will do great harm to young and old alike. I ask for a 
directed verdict of guilty as charged. 

Epilogue 

The judge found this a difficult case. After re-reading On the Sublime he con- 
cluded that Longinus had made grand and unverifiable claims. On that basis he 
found him guilty of fraud. He was also troubled by the author’s evident hostility 
to the imperial regime and to the values of an affluent and pleasure-seeking 
society. But he doubted that Longinus’ critique would truly corrupt the young. 
In lieu of exile he sentenced him to many years of community service as night 
watchman in the local library. Even now late at night one can sometimes hear 
exclamations of delight, when in the intervals between his solitary rounds he 
finds yet another example of the sublime. 


NOTES 

1 Unless otherwise indicated, I use the translation of W. Hamilton Fyfe in 
the Loeb Classical Library. 

2 Cf. 12.1. 

Q 

0 The evidence about such treatises was gathered by Ludwig Radcrmacher, 
Artium Scriptores (9-10). 

^ The surviving portion of an ars rhetorica ascribed to a Longinus is now 
accessible in Patillon and Brisson, fr. 48. 

^ This reading is based on a conjecture, an inspired one, by Rothstein, for 
the bland and inappropriate manuscript reading, “she was frightened.” 

® The “Byzantine University” entry in Wikipedia provides a useful discussion. 

2 See the “Aelius Theon” entry in Wikipedia and the “Aphthonios of 

Antioch” entry in the Oxford Dictionary of Byzantium for more information. 
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APPROACHING THE INEFFABLE: FLOW, SUBLIMITY, 
AND STUDENT LEARNING 

DONNA H El LAND 


“I GET IT!” “THAT’S IT!” “I FIGURED IT OUT!” Anyone who has ever 
focused unwaveringly on solving a problem, anyone who has all at once 
understood the real stakes of an issue or an argument, anyone who has sud- 
denly answered a seemingly unanswerable question will understand the excite- 
ment of those magical moments when — apparently out of nowhere — the right 
answer presents itself. And anyone who has ever taken a class, or taught one, 
will know that those moments do not — in fact — come out of nowhere, that 
they are nonetheless elusive in their workings and difficult (at best) to engineer. 
And so the question arises: do such moments have any place in our thinking 
about the goals of education? Is it legitimate to build into a class or program 
or entire curriculum an expectation that students will pursue intellectual 
inquiry with a passion that leads them to insight? 

Not everyone thinks so. Take, for example, Alan Bennett’s play The History 
Boys, which is about a group of teachers preparing a class for the Oxford / 
Cambridge entrance exams. At stake is what they teach and how, with the inspi- 
rational but decidedly unorthodox methods of Hector being played off against 
those of the sexy, young Irwin and the not sexy, not young, but ever solid Mrs. 
Lintott. The play prevents us from easily championing Hector’s mode of teach- 
ing by also making him something of a sexual predator, but as far as debates 
about educational goals, methods and outcomes go, the headmaster — less than 
inspirational though he may be — sums it all up well when he says: 

Shall I tell you what is wrong with Hector as a teacher? 

It isn’t that he doesn’t produce results. He does. But they are 
unpredictable and unquantifiable and in the current educational 
climate that is no use. He may well be doing his job, but there is 
no method that I know of that enables me to assess the job that 
he is doing. 

There is inspiration, certainly, but how do I quantify that? (67) 1 

The word “inspiration” here is resonant, meaning — in its Latin roots — to 
“breathe . . . into”; to “infuse some thought or feeling into,” especially “by 
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divine or supernatural agency” (one thinks of poets being inspired by their 
muses); to “arouse, awaken” or even “kindle” “ in the mind or heart” 
(“Inspire,” def. 1. 1 , II. 4, 4a, 5b). The notion of inspirational teaching as “kin- 
dling” students’ intellects calls to mind language often used by W. Robert 
Connor, who — when he was president of the Teagle Foundation — liked to 
characterize a good college experience as one that helped students “catch 
fire.” While one might smile at a phrase that conjures Monty Python-like sce- 
narios of students aflame, it is also clear what he meant by it. He was talking 
about the same kind of inspirational experience that Hector aimed to conjure 
for the history boys, the same kind of intense engagement leading to insight 
that is the subject of this essay. And in an argument that complements 
Connor’s essay in this collection — even as it counters Bennett’s headmaster — I 
want to argue that the inspirational and the assessable are not so much 
opposed as complementary: the inspirational need not be as unpredictable as 
the headmaster implies, even as the assessable need not be as reductive. 

In making this argument, I am countering not only Bennett’s headmaster 
as he dismisses a pedagogy he views as unassessable, but also those on the 
other side of this debate, who fear assessment methodologies that can not cap- 
ture the fullness, the subtlety, the “ineffability” of genuine learning experi- 
ences. Here I am encouraged by the work of Elaine Showalter, whose book 
Teaching Literature formulates both a specific overall learning goal for the field of 
literary study (“to train our students to think, read, analyze, and write like lit- 
erary scholars, to approach literary problems as trained specialists in the field 
do, to learn a literary methodology ...”) and a set of common “competencies 
and skills” that “we want students to learn” (25-26). " One could agree or dis- 
agree with her formulation, but the very fact of its existence makes clear that 
literature classes and classrooms are built around quite specific goals (they just 
might not be shared). Creating “catch fire” experiences for her students is not 
one of Showalter’s, but if we hold on to the idea — if we assume that experi- 
ences of intense engagement leading to insight can be at the heart of a class- 
room learning experience — we can still ask: are they truly as elusive as all 
that? Do we really have no way at all to talk about them, understand them, 
maybe even shape them to some extent? 

I contend that we do, and in what follows, I make this case in an argu- 
ment that develops through several stages. Beginning with the concept of 
“flow” in psychology, I build on the work of L. Dee Fink to argue that flow 
experiences can help shape particularly intense forms of student engagement 
in learning, and move on to consider such engagement as not only affective 
but also as cognitive and even creative experience. Proceeding to the second 
stage of my argument, I contend that aspects of flow experiences are analo- 
gous to — perhaps even synonymous with — the experience of the sublime, 
which has been powerful within the field of literary study, and again consider 
the affective, cognitive and creative dimensions of that experience. Finally, I 
turn to the question of whether sublime experiences can be not only inten- 
tionally shaped in a classroom setting but also assessed. Drawing on existing 
research that points to the possibility of accomplishing this seemingly impossi- 
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ble task, I consider what it would take to develop such an assessment mecha- 
nism — one that would gauge the extent to which students engage so intensely 
in their work as to achieve not only affective but also cognitive results — and 
raise the possibility that disciplinary experts and assessment experts could 
work together on such a project. 0 This kind of assessment could tell us a great 
deal about when and why students experience the remarkably intense form of 
engagement and learning that is the focus of this essay, and so make it a more 
regular and replicable part of their education. 

In experimenting with this approach to understanding student learning, I 
aim first of all to advance the discussion of teaching and learning in the litera- 
ture classroom. Further, I hope to move discussions of disciplinary assessment 
in a direction somewhat different from — and I hope complementary to — those 
that we have and are developing. Good disciplinary assessment is certainly tak- 
ing place at institutions across the country, and one can learn a great deal 
from current scholarship and reporting on this subject. 4 What seems still 
underdeveloped in the literature and practice of assessment are efforts to 
bring the language and tools of a specific discipline to bear on the assessment 
of student learning in that discipline. In experimenting with such an 
approach, my intention is to explore the extent to which a disciplinary vocabu- 
lary can energize and help with the work of assessment, and perhaps also 
bridge the gap between scholars / scholarship in a field such as literary study 
and in the fields of institutional and educational research. 

“Flow” and Student Learning 

I begin outside the discipline of literary study, with the work of psycholo- 
gist Mihaly Csikszentmihalyi, whose notion of flow — as L. Dee Fink has 
argued before me — speaks implicitly to the work of classroom teaching (Fink 
152-54). Csikszentmihalyi developed the concept of flow from his study of 
“people who seemed to be doing things that they enjoyed but were not 
rewarded for with money or fame.” He worked with “[cjhess players, rock 
climbers, dancers, and composers [who] devoted many hours a week to their 
avocations,” and asked: 

Why were they doing it? It was clear from talking to them that 
what kept them motivated was the quality of experience they felt 
when they were involved with the activity. . . . [I] t often involved 
painful, risky, difficult activities that stretched the person’s capaci- 
ty and involved an element of novelty and discovery. This optimal 
experience is what I have called flow, because many of the 
respondents described the feeling when things were going well as 
an almost automatic, effortless, yet highly focused state of con- 
sciousness. (110) 

Further, he writes, “[t]he flow experience was described in almost identi- 
cal terms regardless of the activity that produced it. Athletes, artists, religious 
mystics, scientists and ordinary working people described their most rewarding 
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experiences with very similar words. And the description did not vary much by 
culture, gender, or age ...” (110). The nine repeatedly identified qualities of 
the flow experience are: 

• “There are clear goals every step of the way.” 

• “There is immediate feedback to one’s actions.” 

• “There is a balance between challenges and skills.” 

• “Action and awareness are merged.” 

• “Distractions are excluded from consciousness.” 

• “There is no worry of failure.” 

• “Self-consciousness disappears.” 

• “The sense of time becomes distorted.” 

• “The activity becomes autotelic.” (111-13) 

This list begins with what one might describe as the structure of the experi- 
ence: “clear goals,” “immediate feedback,” and “a balance between skills and 
challenges,” all of which are reasonably easy to create and control. As one 
moves down the list, though, the characteristics of flow become ever less easy 
to structure: “action and awareness are merged,” “distractions” disappear, 
along with “the worry of failure.” That merging of “action and awareness” 
signals a blurring of boundaries that becomes more intense in subsequent 
items, and by the time one reaches the point of self-consciousness disappear- 
ing, the flow experience seems to describe the subject’s relationship to what 
s/he is doing in a way that signals not just engagement, but an absorption by 
one’s tasks that might even be described as a form of self-transcendence (or 
self-loss). Describing what it means for self-consciousness to disappear, 
Csikszentmihalyi writes: “[Ajftcr an episode of flow is over ... [w]e might 
even feel that we have stepped out of the boundaries of the ego and have 
become part, at least temporarily, of a larger entity. The musician feels at one 
with the harmony of the cosmos, the athlete moves at one with the team, the 
reader of a novel lives for a few hours in a different reality” (1 12-13). The dis- 
torting of time goes hand in hand with this dissolution of self, and the sense of 
the activity itself as autotelic or “an end in itself” reinforces this understand- 
ing of the flow experience as something that takes one out of oneself. 

To this point, flow experiences would seem to be the province of individu- 
als. Fink has noted that such experiences can perhaps be created in the class- 
room, arguing that “if teachers design their instruction properly, they can cre- 
ate the conditions in which flow activities are likely to occur” (154). In so 
doing, he picks up on an important aspect of Csikszentmihalyi’s thinking, 
which positions flow as an aspect of creativity, and sees creativity not in “tradi- 
tional” terms, as something that characterizes individuals, but as a process that 
takes place within a system. 5 That system involves not only the individual, but 
also an established “domain” (a specific knowledge base “nested” in a larger 
“culture”) and “field” (his term for the “gatekeepers” that allow the knowledge 
base to change) (Csikszentmihalyi 27-28). The classroom “system” certainly 
modifies the one Csikszentmihalyi describes, for a student’s insight will not 
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necessarily constitute an original and enduring contribution to a domain, but 
it is still valuable to consider that moment of insight in the context of the 
classroom “system” in which the established domain is shaped and advanced 
by the teacher and ideally by peers as well (the “gatekeepers”). This is the cre- 
ative work of teaching and learning in a collaborative environment. 

Is it realistic to think colleges and universities could foster such a para- 
digm for teaching and learning, and even if they did, would it really be effec- 
tive? Here it is helpful to relate the notion of flow — understood as a process of 
intense engagement in a task or activity that, I would argue, ultimately leads 
to insight — with recent research on student learning that also links engage- 
ment with attainment. George Kuh and his colleagues at the National Survey 
of Student Engagement (NSSE) have argued — in their study Student Success in 
College: Creating Conditions that Matter — that 

student engagement has two key components that contribute to 
student success. The first is the amount of time and effort stu- 
dents put into their studies and other activities that lead to the 
experiences and outcomes that constitute student success. The 
second is the ways \sic\ the institution allocates resources and 
organizes learning opportunities and services to induce students 
to participate in and benefit from such activities. (9) 

Like the creativity that is nurtured by the flow experience, student engage- 
ment hinges on the work of an individual in a system (the classroom as 
well as the college or university as a whole), and can open up multiple 
paths to success in college. Kuh et al. offer a number of recommendations 
for creating especially strong forms of student engagement in learning, 
including one that reads like a recipe for creating flow experiences. The 
injunction to “Make Talent Development a Central Tenet in the 
Institution’s Operating Philosophy” advises educators to “[s]et perform- 
ance standards for students at high but attainable levels consistent with 
their academic preparation,” “[pjrovidc generous amounts of helpful, con- 
structive feedback,” “[bjalance academic challenge with adequate sup- 
port,” and “[u]se pedagogical approaches that complement students’ 
learning styles” (300-02). And there is no doubt of the effectiveness of 
engaged learning methods: “engagement increases the odds that any stu- 
dent — educational and social background notwithstanding — will attain his 
or her educational and personal objectives, acquire the skills and compe- 
tencies demanded by the challenges of the twenty-first century, and enjoy 
the intellectual and monetary gains associated with the completion of the 
baccalaureate degree” (Kuh, High-Impact 22). 

If it is fair to correlate experiences of flow with student engagement as 
understood by Kuh and his colleagues (and I would argue that it is, though 
flow is perhaps a more intense form of engagement than most), then fostering 
flow experiences can indeed lead to student success. Still, the question 
remains: what exactly do students gain? Flow experiences are anchored in a 
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series of recognizable intellectual or cognitive moves (articulating goals, offer- 
ing feedback, defining an appropriate level of intellectual reach) and con- 
clude — at least some of the time — with intellectual or cognitive gain. 
Importantly, Csikszentmihalyi elaborates on the “autotelic” nature of flow 
with reference to science, saying: 

Scientists often describe the autotelic aspects of their work as the 
exhilaration that comes from the pursuit of truth and of beauty. 

What they seem to describe, however, is the joy of discovery, of 
solving a problem, of being able to express an observed relation- 
ship in a simple and elegant form. So what is rewarding is not a 
mysterious and ineffable external goal but the activity of science 
itself. It is the pursuit that counts, not the attainment. (122) 

Translating this insight to the literature classroom, one can argue that the 
essence of flow comes in the intensity of the learning process itself, the intensi- 
ty of the engagement with the text and the questions it raises. At its heart is 
the intellectual “pursuit” of learning: “discovery,” “solving a problem,” and 
“express [ing] an observed relationship in a simple and elegant form.” Flow 
thus seems to encompass and drive the rational, the analytical, and to point to 
the generation of such outcomes as being at the heart of the kinds of learning 
described here. Indeed, one might go so far as to say that the engagement cre- 
ated by flow experiences is — at its most developed — not just a means by which 
one moves towards learning, but a learning outcome in itself (a subject to 
which I’ll return later). ® 

Even as flow and the engagement it describes can be seen as enmeshed in 
the work of cognition, they are also are clearly affective as well, and I want to 
argue that affective experiences can also have a place in a classroom. I am 
hardly the first person to have made this claim. Benjamin Bloom and his col- 
leagues’ still influential Taxonomy of Educatimal Goals addressed not only the 
cognitive domain but the affective as well, noting the close connection 
between the two. More recently, L. Dee Fink has argued for a revision of 
Bloom’s cognitive outcomes in particular, observing that “individuals and 
organizations involved in higher education are expressing a need for important 
kinds of learning that do not emerge easily from the Bloom taxonomy, for 
example: learning how to learn, leadership and interpersonal skills, ethics, 
communications skills, character, tolerance, and the ability to adapt to change” 
(29). Further, and importantly, Fink states: 

My interpretation of the aforementioned statements is that they 
are expressing a need for new kinds of learning, kinds that go well 
beyond the cognitive domain of Bloom’s taxonomy and even 
beyond cognitive learning itself. This suggests that the time may 
have arrived when we need a new and broader taxonomy of sig- 
nificant learning. (29-30) 
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The development of that “taxonomy of significant learning” is the subject of 
his book of that name, and it has six major categories: “foundational knowl- 
edge,” “application,” “integration,” “human dimension,” “caring” and “learn- 
ing how to learn.” 8 At a first reading, one wants to zero in on those learning 
experiences that seem particularly focused on outcomes that are other than 
cognitive as those that most obviously revise Bloom’s list: caring, for example, 
and the learning experiences that engage the “human dimension,” which Fink 
describes as “address [ing] the important relationships and interactions we all 
have with ourselves and others” (44). Fink himself does not distinguish cogni- 
tive and other kinds of outcomes so easily, though, guiding us to see this tax- 
onomy as “not hierarchical but rather relational and even interactive,” so that 
“achieving any one kind of learning simultaneously enhances the possibility of 
achieving the other kinds of learning as well” (32). 

Fink’s understanding of cognitive and non-cognitive outcomes as connect- 
ed taps into the same long history of research on the relationship of cognition 
to affect that Bloom invoked, and leads one to see — as others have argued — 
that the distinction between the cognitive and non-cognitive realms is perhaps 
not even sustainable. As early as the nineteenth century, William James argued 
that physical experience can cue a specific emotion without the involvement of 
consciousness, or in other words, that “we feel sorry because we cry, angry 
because we strike, afraid because we tremble” (qtd. in Dawes 455). 9 Literary 
critic James Dawes notes that “contemporary neural science” validates this 
view when it demonstrates that “emotions can overwhelm cognition” (455, 
457), and Antonio Damasio — who makes scientific research on this subject 
available to an audience of non-scientists — pushes even farther when he 
argues that “the reasoning system evolved as an extension of the automatic 
emotional system, with emotion playing diverse roles in the reasoning process” 
(xi-xii). 10 

That feeling and thinking are connected seems at this point indisputable, 
and what this line of thought urges us to consider is the possibility that emo- 
tion — feeling that is grounded in bodily experiences such as crying and striking 
(to return to the examples of my last paragraph) — is not only intertwined with 
cognition, but is actually a form of cognition, a way of coming to knowledge. 
Recent work on what the field of cognitive psychology labels “grounded cogni- 
tion” helps to make this case. In a valuable overview of this work, Lawrence W. 
Barsalou draws together a range of research that challenges the notion that 
cognition is distinct from “perception,” “action” and “introspection” (6 1 7) and 
states: ‘“Grounded cognition’ reflects the assumption that cognition is typically 
grounded in multiple ways, including simulations [of perceptual, motor, and 
introspective states], 11 situated action, and, on occasion, bodily states ” (619, ital- 
ics mine). What this perhaps means for classroom teaching is that students’ “gut 
feelings” and affective responses to texts can be understood as valuable in 
themselves and as pushing into the realm of cognition. 

And here I want to turn from psychology and neuroscience back to the 
discipline of literary study and ask whether we can not get an even better 
understanding of this aspect of classroom teaching by working with a tool that 
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literary critics use often: theories of the sublime. Sublime experiences are 
intense affective experiences whose connection to the work of the intellect has 
been an ongoing subject of discussion. And sublime experience is a valuable 
lens through which to understand those key moments in a literature classroom 
in part because literature itself so often seeks — more or less explicitly — to gen- 
erate them. 

Creativity and the Sublime 

Sublime experiences are helpful in thinking about the teaching of litera- 
ture because they engage exactly that moment — a moment that can arise in 
reading, in teaching, and in learning — when one’s relationship to one’s subject 
is all consuming. Whether one is overwhelmed by a novel, or whether one sud- 
denly sees all its moving parts fall into a kind of order, that interaction is all 
that matters. 1 -^ Characterized by a blurring of boundaries between an individ- 
ual and the world around her that echoes key aspects of the flow experience 
described by Csikszentmihalyi, they might also be described as being at the 
heart of what Csikszentmihalyi called a “traditional” understanding of cre- 
ativity. “The creative process has traditionally been described as taking five 
steps,” he writes. “The first is a period of preparation, becoming immersed, 
consciously or not, in a set of problematic issues that are interesting and 
arouse curiosity”; “[t]he second ... is a period of incubation, during which 
ideas churn around below the threshold of consciousness”; “[t]he third ... is 
insight, sometimes called the ‘Aha!’ moment”; “[t]he fourth component is eval- 
uation, when the person must decide whether the insight is valuable and 
worth pursuing,” and “[t]he fifth and last component of the process is elabo- 
ration. It is probably the one that takes up the most time and involves the 
hardest work” (79-80). 

This is a vision of creativity that focuses not on systems but on individ- 
uals — a vision that Csikszentmihalyi does not discount, but complicates, in 
part by insisting that we see this individual creativity in a systemic context — 
and efforts to account for the “aha!” moment at its heart have, not surprising- 
ly, been many. A line of increasingly scientific research offers explanations 
from fields ranging from psychoanalysis to cognitive neuroscience; insofar as 
theories of the sublime can also be helpful here, the humanities also have a 
role to play in understanding the heart of these moments of insight. 1 '^ 

Theories of the sublime attempt to understand exactly what happens in 
moments of intense engagement between a subject and an object. They can- 
not capture such a moment precisely, but they can approach it — define the 
conditions under which it occurs, the connections between people and things 
that shape and are shaped by it, what the experience looks like before and 
after. In their ability to shadow but not fully capture those moments, theories 
of the sublime are like asymptotic curves, always approaching but never actu- 
ally meeting the lines toward which they seem inevitably headed. Still, they 
have the potential to be powerful levers for pedagogy and that possibility is 
what I’d like to explore, drawing on Edmund Burke’s Philosophical Enquiry into 
the Origin of our Ideas of the Sublime and Beautiful and Immanuel Kant’s “Analytic 
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of the Sublime,” in his Critique of Judgement — still two of the most important 
treatises on the sublime — to do so. 

As theorized by Edmund Burke, sublime experiences erase subject-object 
boundaries (Heiland 33). 14 Burke writes: 

The passion caused by the great and sublime in nature, when those 
causes operate most powerfully, is Astonishment; and astonish- 
ment is that state of the soul, in which all its motions are sus- 
pended, with some degree of horror. In this case the mind is so 
entirely filled with its object, that it cannot entertain any other, 
nor by consequence reason on that object which employs it. 

Hence arises the great power of the sublime, that far from being 
produced by them, it anticipates our reasonings, and hurries us 
on by an irresistible force. (57) 

Sublime experience for Burke is one in which the mind is overwhelmed by an 
external force, and his description of the process suggests considerable passivi- 
ty — indeed, “paralysis of our rational faculty” (Ryan 271) — on the part of the 
person having the experience. 15 Still on the same subject, Burke later cites 
Milton’s “portrait of Satan” from Paradise Lost (I, 589-99) as a source of sub- 
lime experience, writing that “[t]he mind is hurried out of itself, by a croud 
\sic\ of great and confused images; which affect because they are croudcd \sic\ 
and confused” (62). This example is particularly interesting from a pedagogi- 
cal perspective, suggesting that sublime experiences can be sparked by the act 
of reading, but describing them in such a way — as initiated by confusion and 
characterized above all by chasing away rationality — that one has to wonder 
whether such an outcome is in any way desirable in a classroom. Sublimity in 
this form would seem to consist of a turn away from active reasoning to pas- 
sive feeling that would seem very far from what most of us would consider a 
desirable form of student engagement and entirely divorced from learning. 

At the same time, though, Burke’s description of sublime moments — those 
moments of total absorption by something outside oneself — are very close to 
the heart of what Csikszentmihalyi describes as the experience of flow. And as 
Burke probes the “efficient cause” of sublime experiences, he approaches the 
ground so recently mapped by neuroscience, arguing that the mind-body rela- 
tionship is at the heart of those experiences: “Our minds and bodies are so 
closely and intimately connected,” he writes, “that one is incapable of pain or 
pleasure without the other” (129, 133), and he spends considerable time trying 
to discover “what affections of the mind produce certain emotions of the 
body,” as well as “what distinct feelings and qualities of body shall produce 
certain determinate passions in the mind” (129). Vanessa Ryan comments that 
Burke’s “physiologism . . . has invited criticism and ridicule not only in his own 
time but also in our own” (269-70), and a twenty-first-century reader will 
almost certainly smile at his description of how “mimicking the looks and ges- 
tures, of angry, or placid, or frighted [sic], or daring men” has led him to 
experience the very “passion whose appearance [lie] endeavoured to imitate” 
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(Burke 133). 16 Then again, in that description, does Burke not anticipate 
Barsalou’s understanding of “grounded cognition” by over two centuries? 

Here one might want to say, with a nod to Jonah Lehrer, that Burke was a 
neuroscientist, or at least a cognitive psychologist, and anyone in a literature 
classroom — teachers and students alike — will benefit from the historical per- 
spective that Burke brings to discussions of those moments of intense engage- 
ment and insight that are his subject and mine. 17 

Where Burke understands sublime experience as an intense affective experi- 
ence through which one is entirely absorbed by something external to oneself, 
Kant significantly revises this formulation when he identifies sublime experience 
as a function of the human mind above all. “The sublime in Kant is the resist- 
ance against that which had been previously considered sublime,” writes Ryan 
(278). Rather than being overwhelmed by that which is outside itself, the mind 
asserts its superiority, and in that assertion lies the sublime experience. Kant 
explains it like this: “Sublime is the name given to what is absolutely great” (94.§25). 
However, that which is “absolutely great” is “not to be looked for in the tilings of 
nature” (whose parts are all relative to each other), “but only in our own ideas” 
(97. §25). In other words, sublime experience takes place when we understand 
that we will never see absolute greatness in the world around us, but we none- 
theless conceive of such a thing in our minds. And in still other words, sublime 
experience takes place “at the point where pure reason transcends the sensuous” 
(Ryan 278). At that point, the mind not only realizes its own capacity, but under- 
stands itself as independent of that which would overwhelm it and so “saves 
humanity in our own person from humiliation” (Kant 1 1 1 .§28). Sublimity for 
Kant, then, is not about self-loss but about self-assertion and is intimately tied to 
the exercise of our mental capacities. Where Burke’s analysis of the sublime can 
perhaps help us understand something about how affective experience can con- 
tribute to student learning — by framing, for example, what happens when a stu- 
dent loses herself in a text or a painting in a kind of sympathetic identification 
with it — Kant shows us how the individual regains mental control over that kind 
of intense experience, and is thus valuable in understanding how an experience 
of intense engagement with an object of study might be linked back to the 
development of one’s cognitive capacities. His moment of mastery corresponds 
very roughly, perhaps, with the fourth and fifth components of those “tradition- 
al” theories of creativity described above, when the individual evaluates and 
elaborates on the insight of the “aha!” moment. 

My argument has moved from a description of “catch fire” moments in 
student learning, through analysis of the psychological concept of flow to pro- 
vide a framework for understanding the process inaugurated by those 
moments, and finally to a discussion of what I see as the culmination of the 
flow experience: that complete absorption by something beyond oneself that is 
described in the literature of the sublime. Sublime moments, as I read them, 
are without doubt a part of the flow experience — part of what makes creative, 
cognitive insight possible — and perhaps also the insight itself (or the nearest 
we can get to a representation of it). Even as light is both wave and particle, 
sublimity is both experience and insight. 
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In developing this line of thought, I have been trying to come to terms 
with what is often seen as most elusive about the learning experience: what 
engages students in the first place and what happens when their engagement 
leads to those seemingly sudden moments of insight, those moments when 
they magically seem to “get it.” Fink has argued that one can structure classes 
so that students are likely — though not guaranteed — to experience flow in the 
classroom: getting to know one’s students well enough to set goals for them, 
ensuring that those goals are ambitious enough to “stretch” students but not so 
ambitious that they’ll be frustrating, providing “ teaching and learning activities 
that will offer learners the right level of challenge along with proper support” 
(all of these steps relate to what Csikszentmihalyi describes as finding “a bal- 
ance between challenges and skills” [1 1 1]), and providing prompt feedback on 
their work (Fink 152-54). That process can create the utterly focused experi- 
ences that Csikszentmihalyi describes, the loss of self that seems to mimic the 
sublime experiences that Burke and Kant describe, and I would further sug- 
gest that, if one can increase the likelihood of a flow experience, then one can 
also increase the likelihood of a sublime insight or “aha!” moment. And one 
can — and must — develop ways of assessing whether those experiences have 
been achieved and what students have learned as a result. 

Can We Assess Sublime Learning? 1 2 3 ’’ 

Existing assessment methodologies perhaps begin to give us ways to 
understand when and why students experience the kind of learning I have 
described here. Charles Blaich points out that the “Need for Cognition” 

Scale (see fig. 1) — which measures “the tendency for an individual to engage 
in and enjoy thinking” (Cacioppo and Petty 1 16) — offers a possible first step. 
A “tendency” is a “disposition” rather than an outcome, but even a “disposi- 
tion” to the kinds of thinking specified by this particular scale suggests a 
pleasure in immersing oneself in intellectual activity that is reminiscent of 
flow experiences and the sublime learning that I’ve been arguing is at their 
heart. And if this is the case, then what are we to make of the fact that — 
according to data collected through the Wabash National Study of Liberal 
Arts Education — there is a group of students whose “need for cognition” 
actually grows over the first year of college (Blaich)? And that this growth can 
actually be predicted by students’ scores on a group of twelve questions — 
NSSE’s “deep learning” scale 19 — which ask about how frequently or to what 
extent students have: 

1 . Analyzed the basic elements of an idea, experience, or theo- 
ry, such as examining a particular case or situation in depth 
and considering its components; 

2. Synthesized and organized ideas, information, or experiences 
into new, more complex interpretations and relationships; 

3. Made judgments about the value of information, arguments, 
or methods, such as examining how others gathered and inter- 
preted data and assessing the soundness of their conclusions; 
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4. Applied theories or concepts to practical problems or in new 
situations; 

5. Worked on a paper or project that required integrating ideas 
or information from various sources; 

6. Included diverse perspectives (different races, religions, 
genders, political beliefs, etc.) in class discussions or writing 
assignments; 

7. Put together ideas or concepts from different courses when 
completing assignments or during class discussions? 

8. Discussed ideas from [their] readings or classes with faculty 
members outside of class; 

9. Discussed ideas from [their] readings or classes with others 
outside of class (students, family members, co-workers, etc.); 

10. Examined the strengths and weaknesses of [their] own views 
on a topic or issue; 

1 1. Tried to better understand someone else’s views by imagining 
how an issue looks from his or her perspective; 

12. Learned something that changed the way [they] understand 
an issue or concept. 20 


In other words, the twelve educational experiences identified in this group of 
questions make it more likely that students will experience a greater “need for 
cognition,” and — again — if we can even speculatively link the “need for cog- 
nition” with the intensely engaged flow experiences that I have been discussing 


1. I would prefer complex to simple problems. 

2. I like to have the responsibility of handling a situation that requires a lot of thinking. 

3. Thinking is not my idea of fun.* 

4. I would rather do something that requires little thought than something that is sure to challenge my 
thinking abilities.* 

5. I try to anticipate and avoid situations where there is likely a chance I will have to think in depth about 
something.* 

6. I find satisfaction in deliberating hard and for long hours. 

7. I only think as hard as I have to* 

8. I prefer to think about small, daily projects to long-term ones* 

9. I like tasks that require little thought once I've learned them* 

1 0. The idea of relying on thought to make my way to the top appeals to me. 

11. I really enjoy a task that involves coming up with new solutions to problems. 

12. Learning new ways to think doesn't excite me very much.* 

13. I prefer my life to be filled with puzzles that I must solve. 

14. The notion of thinking abstractly is appealing to me. 

15. I would prefer a task that is intellectual, difficult, and important to one that is somewhat important but 
does not require much thought. 

16. I feel relief rather than satisfaction after completing a task that required a lot of mental effort.* 

1 7. It's enough for me that something gets the job done; I don't care how or why it works.* 

18. I usually end up deliberating about issues even when they do not affect me personally. 

'Reverse scoring is used on this item. 

Fig. 1 . 18-Item Need for Cognition Scale, Cacioppo et al. “The Efficient Assessment of Need for Cognition” (307). 
Reprinted with permission of the authors. 


126 




APPROACHING THE INEFFABLE: FLOW, SUBLIMITY, AND STUDENT LEARNING 


(and at this stage this line of thought is no more than speculation or sugges- 
tion, an invitation to research), then we are on the road to assessing the fre- 
quency and effectiveness of sublime learning in a classroom. 

Blaich’s data map one possible path to assessing whether we are creating 
the conditions for flow experiences and sublime learning. His research has a 
complement in that of Thomas Nelson Laird and his colleagues, who have 
shown that those same NSSE questions cited above correlate positively with 
students “critical thinking dispositions” (Nelson Laird et al., “Predictive 
Validity”; Nelson Laird, “Unpacking”) which Nelson Laird describes as “much 
like need for cognition” (E-mail to author). Both Blaich and Nelson Laird, 
then, are pointing to practices — those identified by NSSE’s deep learning 
scale — that help make flow experiences and sublime learning possible, and still 
more direct measures of those outcomes can be imagined. One might ask, for 
example: “Have you ever been so totally absorbed in a book / poem / class 
discussion that you lost track of time?” Given that such experience can lead to 
cognitive insight (solving a problem, seeing a truth) that in turn should lead to 
a sorting, sifting and development of ideas of a recognizably academic sort, 
we could also inquire about the follow-up: “Did that experience give you 
insight into the central ideas of the book / poem / discussion” and “Were you 
were able to develop that insight in a class discussion / paper / exam”? In this way, 
we would begin to develop a way of measuring the seemingly unmeasurable, of 
furthering our understanding of that which has seemed to define an outer limit 
of what can be articulated and understood about learning.? 1 

And so I conclude with a question: can we go still further down this road? 
If the answer is yes, then I would also ask whether those who would argue for 
this intensely engaged form of learning as a crucial part of undergraduate 
education could combine forces with those whose expertise would help us 
assess its effectiveness. Gould disciplinary experts collaborate with assessment 
experts to develop a way of gauging whether students are experiencing flow 
and gaining the insight that can come with it??? Such learning is not just a 
means to an end but also an end in itself, a learning experience and learning 
outcome rolled into one, and while I do not know of a single instrument that 
measures both engagement and cognition — the linked characteristics of flow 
experiences and of sublime experience that have been the focus of my argu- 
ment — that is just what is needed here.? 3 Such an instrument could surely 
help us as we work to create sublime learning experiences and to assess exactly 
what is gained through them. That gain would certainly include the lessons of 
the task at hand (that is, the exercise around which the flow experience is 
structured), as well as the methods and subject matter of the discipline(s) in 
which the work is situated, to some degree at least, and may even reach more 
widely. Csikszentmihalyi links experiences of flow to happiness, and an educa- 
tion that can foster happiness — a happiness tied to learning — can sound 
cliched but is surely a good thing. Such an education speaks to the whole per- 
son, and to a central goal of liberal education today. To the extent that the 
teaching of literature can contribute to those — even help to shape them — both 
the discipline and the larger project of liberal education benefit. 
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NOTES 

1 These remarks on The History Boys are drawn from a presentation I initial- 
ly made to the Council of Independent Colleges in 2007, and have 
repeated in other talks, for the American Philological Association, in 

2008, and the American Council of Learned Societies, in 2009. 

9 

Thanks to Steven Mintz, Director of the Graduate School of Arts and 
Sciences’ Teaching Center at Columbia University, for making me aware 
of Showalter’s work on this subject. 

^ Thanks to Charles Blaich, Director of Inquiries at the Center of Inquiry 
in the Liberal Arts at Wabash College and Executive Director of the 
Higher Education Data Sharing consortium (HEDS), for directing me to 
the concept of “flow” and for invaluable help in thinking through some of 
the issues I engage in this essay. 

^ See especially Banta’s Assessing Student Learning in the Disciplines (a volume 
that reprints articles from Banta’s Assessment Update) and the program, 
departmental and disciplinary efforts catalogued by the National Institute 
of Learning Outcomes Assessment (NILOA). 

^ Cole’s essay in this volume also discusses this point. 

0 Thanks to Charles Blaich for helping me understand this point. 

^ Bloom and Inis colleagues’ taxonomy of cognitive skills (published in 1956) still 
resonates today, mapping out development from simple to complex ways of 
knowing: knowledge, comprehension, application, analysis, synthesis and eval- 
uation. Their discussion of affective goals, which appeared eight years later, is 
focused on the development of an internalized system of values via a five-step 
process that begins with “receiving” of “stimuli,” moves through “respond- 
ing” to them, “valuing” them, “organizing the values into a system” and final- 
ly “reach [ing] a point where the individual responds very consistently to 
value-laden situations with an interrelated set of values, a structure, a view of 
the world” (Krathwohl et al. 34-35). The second taxonomy has clearly not 
sparked the same interest as the first, perhaps for the very reasons that the 
authors identify when they discuss the “[ejrosion of [ajffective [vjalues” in the 
articulated goals of a number of courses he and his colleagues studied: diffi- 
culty in measuring such outcomes, the time it seems to take to reach them, 
the “privacy” of one’s personal values, and the fear that an education focused 
on affective outcomes would amount to “indoctrination” (Krathwohl et al. 16- 
18). That said, Bloom and Inis colleagues’ identification of these two tax- 
onomies is valuable, as is their acknowledgement of the fact that “cognition 
and affect can never be completely separated,” and “the possibilities that one 
is in large part the effect of the other” (Krathwohl et al. 85). 

^ Fink introduces his taxonomy on pages 30-31 and works with it through- 
out the book. 

^ Krathwohl et al. discuss James’ argument for the connection of the affec- 
tive and cognitive domains as well (46-47). 

For more on the line of thought that extends from James through 
Damasio, see Jonah Lehrcr’s discussion in Proust 15-22. 
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1 1 In this context, simulation is “the reenactment of perceptual, motor, and 
introspective states acquired during experience with the world, body, and 
mind” (Barsalou 618). 

1 - Connor’s essay in this volume offers a complementary perspective on the 
sublime. 

13 Csikszentmihalyi discusses major lines of research in this area, pointing to 
(a) psychoanalytic explanations that see “the curiosity at the roots of the 
creative process — especially in the arts” as “triggered by a childhood expe- 
rience of sexual origin, a memory so devastating that it had to be 
repressed” and “the creative person” as “one who succeeds in displacing 
the quest for the forbidden knowledge into a permissible curiosity,” using 
the “incubation” period to tap into subconscious concerns that are at the 
heart of the creative endeavor (101); and (b) research based in cognitive 
theory, which also “assume [s] ... that some kind of information process- 
ing keeps going on in the mind even when we are not aware of it,” but 
views connections between ideas as forming “more or less randomly,” with 
those that are “robust surviving] long enough to emerge eventually into 
consciousness” (101). He himself offers a third possibility, speculating that 
the distinction between “serial and parallel processing of information” 
may provide an analogy for how the brain seeks to solve problems, break- 
ing them up into component parts so that it can work on them separately 
and in non-linear fashion (110-11). 

More recently, research in cognitive neuroscience has taken strides to 
understanding exactly how “aha!” moments work. Summarizing this 
work, Jonah Lehrer argues that the “insight process ... is a delicate mental 
balancing act” that begins with focused “attention on a single problem” 
but then demands a time of “relaxation” (as researcher Mark Jung- 
Beeman, qtd. in Lehrer, put it), of “letting the mind wander,” which cre- 
ates the neurological conditions for insight to occur (“Eureka Moment” 
43). That time of wandering is interestingly accommodated within the 
flow experience. Csikszentmihalyi quotes Freeman Dyson identifying the 
moment of relaxation, “shaving or taking a walk” as part of what allows 
the “merging of action and awareness” in the flow experience (1 19). Such 
moments open the mind to “unconventional ideas” and to the sudden 
“burst of brain activity” that, we now know, inevitably accompanies the 
“aha” moment (“Eureka Moment” 43), and are — I would argue, extrapo- 
lating from what Csikszentmihalyi writes — quite different from the sorts of 
distractions (thinking about “health or tax problems,” for example) that 
disrupt the process of flow (Csikszentmihalyi 1 12). 

14 In my discussion of Burke and Kant, I work with some of the ideas and 
that I developed in Gothic and Gender: An Introduction, making some of the 
same points and moving beyond/ revising them on occasion. 

I 3 Vanessa Ryan reads Burke much as I do, arguing that “Burke minimizes the 
role of the mind in the experience of the sublime and that he characterizes 
the sublime as a natural force that is by its very definition beyond man’s 
ability to control” (267). She argues that a history of “[rjeading Burke from 
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a Kantian perspective has led critics to deemphasize the physiological basis 
of Burke’s theory and has given rise to the view that he associates the sub- 
lime with an act of mastery and a sense of self-exaltation” (270). 

As Ryan also notes, Thomas Weiskel comments on this aspect of Burke’s 
writing on the sublime (270). 

Lehrcr’s Proust Was a Neuroscientist makes a persuasive case for the ways 
that literature anticipates the development of neuroscience, and his open- 
ing chapter on Walt Whitman specifically takes up the question of mind 
to body, and of feeling to thinking. 

“Sublime learning” is a phrase — and an idea — with a history, both of 
which I have pondered for some time. Laura J. Rosenthal and I used “sub- 
lime learning” in our 2007 “Request for Proposals” to this volume, and W. 
Robert Connor engages the idea in the Teagle Foundation’s Liblog entry 
entitled “Magical Rationalism” (Sept. 9, 2009). 

1 Nelson Laird, Shoup and Kuh discuss the development of the NSSE deep 
learning scale and place the scale in the context of other research on “deep 
learning.” In contrast to “surface-level processing,” in which students 
“focus on the substance of information and emphasize rote learning and 
memorization techniques,” “deep-level processing” is “focus [ed] not only 
on substance but also the underlying meaning of the information.” Deep 
learning is active and engaged, “represented by a personal commitment to 
understand the material which is reflected in using various strategies such 
as reading widely, combining a variety of resources, discussion [of] ideas 
with others, reflecting on how individual pieces of information relate to 
larger constructs or patterns, and applying knowledge in real world situa- 
tions.” Finally, “deep learning” involves “integrating and synthesizing infor- 
mation in ways that become part of one’s thinking and approaching new 
phenomena and efforts to see things from different perspectives” (3-4). 

See Nelson Laird, Shoup and Kuh for this table (24). Note that the table 
includes three additional items which were dropped, due to space con- 
straints, in 2005. 

O 1 

In this volume, Walvoord’s essay also arrives at a formulation of questions 
along these lines — numbers 8 and 9 of her list of proposed learning goals 
for undergraduate literature majors, and Sarah Goodwin’s essay speaks 
directly to the question of sublime learning and its assessability. Charles 
Altieri’s essay resonates with these and with mine in interesting ways. 

-- Rachelle Brooks’ study of liberal education outcomes in the disciplines of 
classics and political science (discussed in this volume) begins to model this 
kind of collaboration, though the outcomes on which that study focuses 
are critical thinking and post-formal reasoning. 

73 While no one instrument assesses both engagement and cognition, Nelson 
Laird has commented that “ [d] eep approaches to learning, as a construct, 
likely sits between common measures of engagement (time on task, fre- 
quency of contact with faculty, etc.) and the ‘intense engagement’” that is 
my focus (personal communication). 
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SARAH WEBSTER GOODWIN 


Measurement 

[a] 

If it were not for the Poetic or Prophetic character, the 
Philosophic & Experimental would soon be at the ratio of all 
things & stand still, unable to do other than repeat the same dull 
round over again. 


MU 

Reason or the ratio of all we have already known is not the same 
that it shall be when we know more. 

William Blake, There Is No Natural Religion (1-2) 1 


IN THIS CRYPTIC WORK, BLAKE famously and obliquely makes a case 
for the “Poetic or Prophetic character,” which he understands as a mental 
capacity that can be at work in any kind of thought or expression. Our minds, 
he argues, are capable of learning that is not bounded by what we perceive in 
the material world or by what is already known and taught. Learning can 
grow beyond what is known, and when it does, it has a poetic and prophetic 
character: a shaping force and an intimation of things to come. 

Philosophy and experimental science, he implies, may stop at repeating the 
logic of what is known; it is the power of science to be predictably reproducible. 
But he also implies that not only in poetry but also in philosophy and all of the 
known sciences, a certain kind of learning may happen that breaks the bound- 
aries of what is known — that, we would say now, changes paradigms, or intro- 
duces a depth or complexity to matters once thought self-evidently clear. This is 
where the romantic notion of the scientist as discoverer reclaims science as a 
sublime undertaking: thus Keats, in “On First Looking into Chapman’s Homer,” 
compares reading Homer to an astronomer discovering a new star, or to “stout 
Cortez” charting a new continent. 2 

That kind of learning may seem to have little to do with undergraduate 
education and learning outcomes assessment. It could be argued that yes, our 
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philosophers and poets and scientists arc directing their work toward break- 
throughs: new solutions to old problems, new forms of expression, new mole- 
cules and technologies; but our undergraduates learn the foundations of the 
disciplines, basic knowledge, principles, syntagms, conceptual and integrative 
tools and models. We don’t expect breakthroughs or new paradigms from 
undergraduate students, so they are not typically in our learning goals and our 
assessment rubrics. In the influential “Essential Learning Outcomes” devel- 
oped by the Association of American Colleges and Universities (AAC&U) 
under the auspices of its “Liberal Education and America’s Promise” (LEAP) 
initiative, “Critical and creative thinking” is listed among several other 
“Intellectual and Practical Skills”; the culminating outcome is “Integrative and 
Applied Learning, including synthesis and advanced accomplishment across 
general and specialized studies” (AAC&U 3). We sense that this “advanced 
accomplishment” does not encompass Blake’s “Poetic or Prophetic character.” 

Can we imagine Blake’s response to this sense of what we want our stu- 
dents to learn in college? Would he draw a line at the sixteenth year of institu- 
tional education and say, “Now let the real learning begin”? Or might we take 
a cue from Blake’s Songs of Innocence and. Experience, and consider what possible 
role our sense of the “Poetic and Prophetic character” might play in the edu- 
cation of college students? As many colleges undertake to establish goals for 
our students’ learning, devise rubrics based on those goals, and conduct time- 
consuming and sometimes expensive assessments to determine how well our 
students are meeting those goals, we might legitimately ask whether we are 
applying our own disciplines’ most complex hermeneutics to this work. We 
could answer that the learning goal of “creative thinking” that appears on the 
LEAP list of goals and in the rubrics based on them is one way of saying that 
we do ask our undergraduate students to create new knowledge, interpretive 
models, exegeses, and creative works: we apprentice them in, among other 
things, work of a prophetic or poetic character. Their attempts may be as 
Calculus 1 is to higher mathematics — assuming here that higher mathematics 
includes sublime learning — but we aim to get them as far along as we can. 3 
Should we be able to articulate where this learning happens, what it looks like, 
and when it happens at the most advanced level — even when, paradoxically, the 
project is to express something unknown, ineffable, unrecognizable, or mute? 

The project of assessment has its roots in an Enlightenment conception of 
education as profoundly rational: assessment is a form of knowledge-gather- 
ing, of Wissenschcft or research; the object of study is our students’ learning. Its 
research methods presume that this knowledge exists and can be discovered, 
broken down, analyzed, and evaluated piece by piece. We practice this research 
in a post-Enlightenment context that does not allow for easy positivism. There is 
no discipline today in which the status of evidence is not at least somewhat 
contested, the epistemologies at least somewhat unstable. It may be facile to 
assert that the romantic challenge to Enlightenment positivism is still in force 
in our radical questioning of what we can know and discover. But alongside 
that fundamental doubt, so evident in Blake’s assertions above, another 
romantic notion persists that might also seem to thwart the project of estab- 
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lishing goals, rubrics, and assessments for student learning: that is the notion 
that our highest learning is sublime, somehow ineffable, even terrifying, intuit- 
ed as the poetic or prophetic character and best described in metaphor rather 
than abstraction. Many faculty members would likely dismiss such a notion as 
fuzzy, irrational — or romantic. And yet we also recognize that the metadis- 
courses in our disciplines, our foundational theories, nearly always lead to 
irresolvable contradictions or unknowable conditions. Very often, the work 
we admire most addresses or realigns the assumptions under which we pursue 
our learning and, ostensibly, our teaching. Some of us might even argue that 
our ambition for our best students is that they come to understand how fault- 
ridden the foundations of our inquiry are. Thus it seems that our goals for 
their creative thinking are closely aligned with critical thinking: with under- 
standing the ways our knowledge is contingent, fragmentary, and anything 
but disinterested. 

It seems, then, that we may have the rubrics for assessing “the Poetic or 
Prophetic character’' of our students’ work, in the measures of creative and 
critical thinking. And yet there seems to be still too great a disjunction between 
what Blake intends and what rubrics can address. Rubrics, and assessments 
based upon them, surely have more in common with the “charter’d streets” (1) 
and “mind-forg’d manacles” (8) of Blake’s “London” than they do with prolif- 
ic energies. 4 Is it in any way possible to view the rubric’s grid not as a con- 
straint but instead as a means of liberation? 

In this essay, as I develop an answer to that question, I will focus on the 
rubric as a tool for evaluation and assessment, based on my own experience in 
the classroom. I will argue that rubrics are useful; most importantly, they are 
deservedly seen as democratic, in that they make our assumptions and aims 
transparent and accessible to all students. But rubrics have their limitations: 
they may lead us to a false sense of safety, may make us miss openings onto 
new ideas and processes. Rubrics, like our goals for students’ learning and our 
assessments, must be conceived in an ongoing dialogue (explicit or implicit), 
and are themselves subject to evaluation. For them to work over a long-term 
process as part of teaching and learning, they must be expressed in terms that 
encourage both students and faculty to take risks, to go beyond the “same dull 
round”; if we can find the right language, they can also lead both students and 
faculty to be more ambitious, to reach the point of sublime wonder and awe. 
But still, I will argue, if the texts are well chosen and the assignment well 
designed, the best things that come out of the class will exceed the rubric. 

Finally, I will argue that we can extend these lessons from the classroom to 
the role of rubrics in program-level assessments. And that both can be 
Blakeian and liberating in their energies. 

Rubrics 

Rubrics have come to play a crucial role in the assessment of student 
learning. Well-designed rubrics, we know, emerge out of the criteria that we 
actually look for in our students’ best work. They clarify for both faculty and 
students just what it is we are looking for in their learning and achievements, 
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and they lay the groundwork for assessing student learning in the aggregate. 
Although the rubric as an instrument for grading a specific assignment is dis- 
tinct from rubrics that are used for broad and aggregated assessments, both 
work on the same principle: the goals for the students’ learning and achieve- 
ment can be articulated as separate features and can be separately evaluated 
and measured. In grading, the rubric clarifies for faculty and student both the 
goals for the assignment and the criteria for grading. In program-level assess- 
ment, the rubric similarly is based upon the learning goals for the given unit of 
assessment (for example, a course or group of courses in a given discipline), 
and it both clarifies those goals and separates them into assessable units. A 
rubric for an individual assignment can be based upon the larger goals for a 
course or even for a major, and it can simultaneously inform the student, help 
faculty to evaluate the individual work, and lead to aggregated evaluations in a 
program-level assessment. As recent a work as Stevens’ and Levi’s 2005 book, 
Introduction to Rubrics, focuses almost exclusively on the rubric in the classroom, 
though it points out the usefulness of rubrics for broader assessments. For 
rubrics to translate to the programmatic level, faculty must collaborate on cre- 
ating them. The recent work on rubrics edited by Terrell Rhodes and pub- 
lished by AAC&U opens by asking whether this undertaking is even possible: 

Is there a shared set of expectations for learning that individual 
faculty can use in the classroom, that can be aggregated for pro- 
grammatic evaluation and sampled for institutional reporting? . . . 

Gan the shared expectations for learning be articulated so that 
students can use them to understand and make judgments about 
their own learning strengths and weaknesses?... Gan we assess 
student learning in ways that actually provide faculty and stu- 
dents with information helpful to improve pedagogy and the 
development of learning over time as well as provide programs 
and institutions with summative information for reporting? (1) 

The answer, implicitly, is yes, and this publication goes further than any other 
I know of to make the rubrics correspond genuinely to the most complex 
kinds of learning that faculty from multiple disciplines might be willing to 
agree on. In other words, rubrics here go well beyond the context of class- 
room use, and to get there, the faculty’s “shared expectations” become the 
standard. Despite this major difference, the assessment rubric shares with the 
teaching rubric a fundamental structure: it spells out expectations for students’ 
learning in multiple categories that then serve as the basis for evaluation. 

Because my concern here is to scrutinize closely how a rubric works, and 
whether it works as a constraint or a liberating framework, I begin here with 
the rubric’s narrowest scope: the individual assignment. I remember clearly 
the first time I saw a rubric for grading papers; it was in a pedagogy workshop 
well over a decade ago at the college where I teach, and a respected faculty 
member distributed the rubric he used (see fig. 1). He explained that he gave 
the rubric to the students along with the paper assignment, so that they knew 
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what he was looking for; and he filled it out as he graded their papers, so that 
they could see the strengths and weaknesses very clearly. 


1. Responds fully to assignment 

Excellent 

Very Good 

Average 

Fair 

Poor 

2. Expresses its hypothesis, question, 
or problem clearly 

Excellent 

Very Good 

Average 

Fair 

Poor 

3. Begins and ends effectively 

Excellent 

Very Good 

Average 

Fair 

Poor 

4. The content is based on accurate informa- 
tion 

Excellent 

Very Good 

Average 

Fair 

Poor 

5. Provides adequate supporting arguments, evi- 
dence, examples, and details 

Excellent 

Very Good 

Average 

Fair 

Poor 

6. Is well organized and unified 

Excellent 

Very Good 

Average 

Fair 

Poor 

7. Analyzes the data well 

Excellent 

Very Good 

Average 

Fair 

Poor 

8. Uses appropriate, direct language for the 
defined audience 

Excellent 

Very Good 

Average 

Fair 

Poor 

9. Uses adequate sources 

Excellent 

Very Good 

Average 

Fair 

Poor 

10. Correctly acknowledges and documents 
sources 

Excellent 

Very Good 

Average 

Fair 

Poor 

11. Is free of errors in grammar, punctuation, 
word choice, spelling, and format 

Excellent 

Very Good 

Average 

Fair 

Poor 

12. Shows originality and creativity in realizing 
1-9 

Excellent 

Very Good 

Average 

Fair 

Poor 

OVERALL EVALUATION 







Fig. 1 . Rubric for grading papers, author and date unknown. 


I was impressed by the rubric. It clarified things that had seemed murky, 
and I could see that it could help my grading be fair and consistent. Still, 
something about it made me uneasy: it must have been the boxes. They were 
too linear; they seemed to imply that each of these categories was clearly dis- 
tinct from the others, and each could be assessed in an objective, almost scien- 
tific way. So when I made use of it the first time, I took away the boxes, leav- 
ing words on the page and perhaps believing there was a certain ambiguity in 
the white spaces. I found the rubric made my grading easier and quicker. I 
could attend principally to the categories on the sheet of paper, and my stu- 
dents could see just what they needed to revise. I spent less time writing com- 
ments, and achieved much the same thing, I thought, with check marks. 

But very quickly, it seemed there was something missing. I tried to find 
words for it, and it wasn’t easy. It was somewhere in the terrain of number 12: 
“Shows originality and creativity in realizing 1 -9,” and I found that I valued 
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that category disproportionately. For example, some papers that seemed really 
good did not necessarily express a clear thesis or hypothesis in the convention- 
al way at the opening of the paper — and somehow got away with it. It seemed 
that an idea could be developed well with some awkward transitions; that 
sometimes a certain quality took over — something to do with the voice and 
the logic propelling the paper — that seemed even more important than the 
handling of transitions, for example, or the clarity of the conclusion. So I 
added a point to the rubric: “13. Je ne sais quoiT I didn't actually give it that 
much thought at the time; I just wanted the flexibility to be able to acknowl- 
edge a strong paper that broke some of the conventions. 

Not too long after that, I took on an administrative role and stopped 
teaching for a time; the rubric went into a file in the back of the drawer, 
so it didn’t evolve. But that number 1 3 — somehow aptly numbered — 
stayed with me. Why was it particularly satisfying that it was in another 
language that many of my students didn’t understand? That it alluded to 
not knowing rather than to knowledge? Or was this really just intellectual 
laziness, an unwillingness to think through and articulate what happens 
when a student paper has a quality that seems to elude the standard quali- 
ties, almost to rewrite the rules, in a way that seems surprisingly authorita- 
tive or authentic? 

With this question in mind, I’ve looked at rubrics over the past several 
years in hopes of finding one that does articulate this quality. Arguably, that 
number 12 on the first one does: we may well be talking about “originality 
and creativity.” Those are terms out of the romantic lexicon, kin with “Poetic 
and Prophetic character.” Interestingly, most rubrics in my unscientific survey 
do not contain them. (This survey was so unscientific it consisted of slipping 
rubrics into a manila folder, sources unmarked, as I came across them over a 
period of some eight years, little thinking I would ever be writing about them 
and wishing I knew their sources.) They emphasize, to cite another one in my 
file: “Thesis, organization, transitions, development, evidence, conclusion, 
diction & style, mechanics”: all crucial aspects of good writing and clear 
thinking. Another rubric discussed in a workshop at my college a number of 
years ago — again, I no longer know the source — approaches the matter some- 
what differently by providing language to describe whether the given paper 
achieves “High,” “Middle” or “Low” levels of success in four areas: Focus, 
Organization, Style, and Mechanics. Then it concludes with a series of ques- 
tions about the degree to which the author has met these standards (see fig. 
2). The questions make a difference in the tone and function of the rubric. 
Where a grid had suggested something quasi-scientific, a series of earnest 


1 . Does the author have a main idea, and does he or she stick to it? 

2. Does the author make defensible assertions and supply adequate details to support these assertions? 

3. Do the sentences and paragraphs flow smoothly? 

4. Do the sentence patterns vary? 

5. Is the essay relatively free of grammatical errors, punctuation errors, and misspellings? 

Fig. 2. Rubric, author and date unknown. 
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questions implies that there is some kind of dialogue at work. That may 
make this rubric more effective as part of a formative process; but that sense 
of a dialogue is perhaps beside the point when a rubric is part of a summa- 
tive and aggregated assessment. So the shift from abstract criteria to ques- 
tions is not in fact a crucial one, methodologically. And in any case, we rec- 
ognize again the familiar categories on the rubric and the implicit lines 
drawn between them. There is enough of a consensus now about these crite- 
ria that a Google search of the phrase “Thesis, organization, transitions, 
development, evidence, conclusion, diction & style, mechanics” turns up 
37,500 hits, countless numbers of them rubrics from colleges and high 
schools. They are clearly in common use now as a teaching tool and as a 
basis for assessment. In either case, the rubrics that are commonly available 
from writing center websites, English departments and other humanities pro- 
grams bear a family resemblance and, as far as I have seen, do not mention 
sublime learning. 

In the literature about rubrics, we can trace their emergence to the 1 96 1 
publication from the Educational Testing Service that proposed five factors in 
good writing that could be isolated and used for purposes of evaluating writ- 
ing. Bob Broad, in his book, What We Really Value: Beyond Rubrics in Teaching and 
Assessing Writing, quotes them: 

Ideas : relevance, clarity, quantity, development, persuasiveness 

Fornr. organization and analysis 

Flavor, style, interest, sincerity 

Mechanics', specific errors in punctuation, grammar, etc. 

Wording : choice and arrangement of words. (6)^ 

Broad, surveying the effect of rubrics on teaching and assessing writing, 
goes so far as to argue that their limitations have become clear: “The age of 
the rubric has passed,” he says: it gave us efficiency, but at the cost of truth 
(4). His critique of rubrics is founded on two related propositions: that they 
do not reflect what we actually value most in good writing; and that they con- 
tradict the real complexity of knowledge and thought (4). Another important 
critic of rubrics, Brian Huot, has argued that assessment of writing in partic- 
ular has had “roots in a positivist epistemology” (“Toward a New Theory,” 
160) that is divorced from “our understandings about the nature of language, 
written communication, and its teaching” (162). In the place of standardized 
rubrics, Huot has argued for assessment criteria that are holistic and “site- 
based,” that emerge from within specific contexts (162). Like Broad, then, he 
sees student writing and learning as being more complex than rubrics are 
able to capture. Not only are the various features of good writing intercon- 
nected, not only do they exist in shifting hierarchies, but they also depend to 
a surprising degree on the context in which the students are writing and 
learning. And “context” here is a large umbrella covering a wide range of fac- 
tors, from the nature of the assignment and the course to larger factors such 
as the students’ subcultures and social identities, the kinds of academic sup- 
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port available, the nature of the institution, and the prevailing community 
expectations of students’ writing. 

Huot’s and Broad’s critiques are cogent and persuasive. My argument 
here, through the lens of Blake and by implication romantic challenges to 
Enlightenment rationalism, extends these two critiques of the common rubric. 
It may be difficult for any literary scholar working today, schooled in poststruc- 
turalist radical uncertainties, to see the rubric as anything but a flawed fiction, 
at best, and at worst a form of surveillance, in Foucault’s sense of the word: 
the state’s intrusion into learning. At the same time, I am not wholly willing to 
sacrifice the rubric’s usefulness — not just as an instrument for evaluating stu- 
dent work, but as a fundamental principle in teaching: we should make our 
criteria known to our students, and they should be clear and attainable; our 
assessments should emerge clearly from them.® And when we do that, the cri- 
teria probably look a lot like the rubrics quoted above. 

I do, without question, want my students to learn to manage each of the 
aspects of an essay that these rubrics outline. I even catch myself assigning a 
certain moral virtue to them: so, for example, I will share with students my 
disdain when a literary critic willfully disregards evidence in the text, or fails to 
organize an argument in a way that unfolds logically. We spend a lot of time 
talking about what kind of thesis is worth advancing and what kind of literary 
evidence is not only convincing but also accurate, even true. Most days of the 
week, I don’t need more than those points on that rubric to assess my students’ 
work or their learning. And then sometimes I do, and when that happens, it 
can seem as though a crack has opened up and I am not sure what I see 
through it. In the case that follows, an unusual student project led me to recon- 
sider my standard categories of assessment, not only for that assignment but 
for student work in general. I learned that the greater the risks I take with my 
teaching, the more likely the students are to escape the “same dull round” as 
well, and the greater the likelihood that their education and mine seem bound 
with an urgency that is both satisfying and unsettling. To my surprise, I also 
learned that a rubric could help us to reach that point. 

Sergio’s burning shoes 

Recently, I taught a course for the second time that had previously fallen 
flat. The course is in Skidmore College’s interdisciplinary Scribner Seminar 
program, part of the college’s first-year experience. It was one of a cluster of 
eight seminars whose title was “Human Dilemmas.” Of the 1 10 or so students 
in these eight seminars, about a third were admitted through our opportunity 
programs on full scholarship and thus came from disadvantaged backgrounds. 
The “Human Dilemmas” seminars aim to introduce all 1 10 of the students to 
some of the big questions addressed in a liberal arts education, and to show 
them how different disciplinary approaches to them complement each other. 
What can we know? How do we know what we know? What is a self? What is 
the relationship between the self and society? What is the relationship between 
the self and the natural world? These are the overarching questions that frame 
the course. Students read Plato, Peirce, Locke, Darwin, and some less canoni- 
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cal authors who model some approaches to these questions. Although it is not 
a literature course, it stresses the importance of the arts within the context of 
other modes of inquiry, and students write and revise frequent papers that rely 
on the kinds of close readings my students in literature courses also do. 

“Human Dilemmas” is an immensely ambitious course and is notoriously 
difficult to teach, even with small class sizes that allow us to work closely with 
individual students. The texts are challenging and abstract; the students are in 
their first semester at college and are not all interested in epistemological 
uncertainty and Platonic metaphor; and often their papers read like high 
school papers: dutiful and disengaged. Because the students come from an 
extremely broad range of socioeconomic and cultural backgrounds, the class 
can feel disjointed. After struggling through teaching it my first time, I was 
determined to try again and do it better. I was convinced that those big ques- 
tions are inherently compelling, that the material was tough but rewarded 
close attention, and that this course could be a transformative experience for 
first-year students of any background. What seemed essential was to engage 
them from the very first day: to make the course feel more like a quest than a 
canon, to translate the questions into accessible language, to withhold some 
answers, to lend it a sense of adventure, and to keep it moving. 

This involved a number of strategies, most of them pedagogical decisions 
aimed at engaging the students in multiple ways. I instituted a service-learning 
component to the seminar, with grant support from the AAC&U “Bringing 
Theory to Practice” program; I devised classroom activities that prevented 
them from sitting in the same seat every class and compelled them to interact 
and collaborate and talk about the readings in the context of their social iden- 
tities and differences. 

One strategy in particular had major consequences for the students’ final 
papers and for my assessment of them. Our campus has an unusual museum, 
the Tang Teaching Museum and Art Gallery, that actively seeks to engage stu- 
dents and faculty in and through the curriculum. The artist Dario Roblcto 
was due to open a show there early that fall (2008). A catalogue was available 
ahead of time, and something about his work just seemed exactly right for the 
material in the seminar. 7 The curator, Ian Berry, was willing to give the stu- 
dents a preview of the show with a close look at several of the works, and 
Robleto was willing to meet with them when his show was opening. This felt 
like a leap of faith: I couldn’t be sure that the students would respond to the 
works, that the artist would connect with them, that the works would in fact 
seem as closely related to our topics as I thought — or that this experience 
would enhance their learning in any way. But there was something about 
those objects that I thought might speak directly to them, if they, as a class, 
could get past their shared resistance and skepticism. In particular, Robleto’s 
work uses highly charged objects from everyday life that he processes in vari- 
ous ways and then combines into artifacts that work both metaphorically and 
metonymically with a peculiar power. For example, he may grind human 
bones or vinyl LPs to a powder and then bind the powder so that he can 
sculpt objects. He may introduce “authentic” objects, the detritus of past wars 
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or personal events: bullet casings, letters, hair, objects from his family, uniform 
buttons. The artifacts' titles list their components, and read almost like poems. 
His works in this particular exhibition had a common thematic thread about 
war, its personal and societal costs and dilemmas. Robleto is unafraid of affect, 
and looks for an emotional response in the viewer. This quality, combined with 
the sheer originality, beauty and creepiness of his works, makes them immedi- 
ately accessible. 8 

Taking Robleto ’s objects as a starting point, the final assignment for my 
seminar asked the students to create objects of their own, using both metaphor 
and metonymy to communicate about one of the dilemmas we had studied; 
they also wrote a paper documenting and analyzing their objects in the light of 
some of the texts we had read together. They were asked to “process” at least 
one of the components in a way that could be read metaphorically: to pulver- 
ize, cook, chew, weave, burn, dissolve, glue, or otherwise change it, and to read 
that change as a mental as well as physical process. They took this part of the 
assignment on with such enthusiasm that one of my challenges became to save 
some of their more precious objects — ancestral documents, old love letters 
and photographs — from being destroyed. My hope in writing this assignment 
was that working with objects that are intrinsically powerful to them would 
help them to understand in more vivid terms the abstractions that we had 
been grappling with in the readings. Further, I was gambling that Roblcto’s 
works are themselves so provocative, and the museum setting so dramatically 
visual , that many of them would step into that space with some excitement and 
energy that could translate to their work. 

I’m describing this assignment at length in part because I took such pleas- 
ure in designing it: it was, for me, unorthodox, exploratory, risky — maybe 
even, in Blake’s sense, Poetic. As for the students’ projects, let me begin by 
describing one. Sergio Hernandez took a pair of his shoes, filled them with 
dirt mixed with dead leaves and various shredded materials, took them out- 
side in the snow, and lit them on fire. He filmed the shoes burning on his cell 
phone, then created a PowerPoint file with the burning shoes and an Avril 
Lavigne soundtrack, her song “Innocence.” Sergio, the son of Mexican 
migrant workers and an extremely bright, motivated student, had said once 
in class, “If you’re Mexican, everyone assumes that you walked here.” That 
sentence came back to me when I watched his shoes burning. These were 
shoes of his from high school; they did not have an extraordinary history, at 
least none that he told us about. But I knew that he felt already, after four 
months of college, that he had changed a lot, and that the readings we had 
done for our seminar had been unsettling for him (as for all the students). 
When he talked about the project, he said, “Shoes are like knowledge. You 
walk in them a way, and then they wear out and you move on to a new pair.” 
His paper elaborates on that point, with a particular focus on epistemology 
and Charles Sanders Peirce. 

Without dramatizing this point too much, I want to describe what it feels 
like to watch Sergio’s shoes burning. Nothing about this film clip is captured 
well by the rubrics I have quoted above, but I do not know that I have ever 
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seen quite such a powerful piece of student work. What they call to mind is 
again from Blake: 

Tygcr Tyger, burning bright, 

In the forests of the night; 

What immortal hand or eye, 

Could frame thy fearful symmetry? 

In what distant deeps or skies 
Burnt the fire of thine eyes! 

On what wings dare he aspire? 

What the hand, dare sieze the fire? (24) 

It is all but impossible for me to experience these lines the way I did when I 
first read them some forty years ago, but the matter of them here is still famil- 
iar. It’s what Emily Dickinson means when she says that poetry is what takes 
the top of her head off. 9 And that image itself echoes Job 4.13-17: “Then a 

Spirit passed before my face: the hair of my flesh stood up Morton 

Paley (546) quotes this passage from Job in his reading of “The Tyger,” and 
goes on to comment on the sublime force of fire in this poem: “The destruc- 
tive fire of Wrath is also the energy of purification Blake’s furnace is a per- 

petual source of power for transforming a dead world” (550). 11 Because trans- 
formation also means loss, because a new world also means the loss of an old 
one, to look at this power squarely and acknowledge its force may be to fear it 
as much as to welcome it. Sergio’s burning shoes, crackling with flames, posed 
in a slightly vulnerable way with one toe touching the other, can seem both 
terrifying and beautiful. 

Sergio’s artifact is not alone in conveying such intensity; the students were 
remarkably engaged in producing these objects, and they created some star- 
tling, eerie, thought-provoking artifacts. Zachary Peyser filled a milk bottle 
from his grandmother’s farm with soil from his future grave site and surround- 
ed it with a triptych of x-rays of his skull. Kristin Zhou, a first-generation 
Chinese-American, used a scale to “weigh” the relative importance of her two 
cultures, their currencies and their music, burning two CDs until they melted 
into one another at the overlap. Stephen Bissonnette made a paper boat out of 
his Selective Service letter and put in it a rosary and wallet that his great- 
grandparents had brought when they emigrated from Portugal. The students’ 
papers were uneven, reflecting to some extent their varying pre-college prepa- 
ration; but all of them represented profound engagement with some of the 
central issues in “Human Dilemmas.” It seemed that the one point on that 
first rubric that did apply, universally, was “shows originality and creativity.” 
For most, I would need to go further and add something like the “Je ne sais 
quoi ” category, except that the phrase is too flip for the earnestness of the stu- 
dents’ work, and the awe that I still feel looking at photographs of it. This 
project went well beyond a “certain something” in the students’ creativity; 
they were looking backward at who they had been, and forward at who they 
were becoming, in a way that seemed urgent to them. 
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And all of this feels inadequate as a description, because I cannot capture 
the way that the project culminated a semester of challenging readings, discus- 
sions that were sometimes uncomfortable, many moments of resistance or 
unhappiness, much slapdash work and skepticism, and occasional glimpses, rev- 
elations, of how it all fit together in their minds. That context is essential to 
understanding and conveying what is at work, for example, in the pair of burn- 
ing shoes. Recent scholarship on contexts and their role in rubrics and assess- 
ment provides a conceptual framework for understanding just how essential — 
but also complex — contexts are. When, for example, a student who is bi-cultural 
and for whom English is a second language is struggling to express herself, her 
“errors,” in the context of this assignment, can also be read as signs of her 
lived experience, her courage and her progress. In a sense, each student’s con- 
text requires — and to some extent surely receives from the faculty, even if we 
have trouble articulating it — a private rubric that parallels the public one. 1 2 

In part because this assignment was as foreign to me as it was to my students, 
I created a rubric for it and distributed it beforehand. They had seemed puzzled 
by the assignment, nervous about how it would be graded, and hesitant to launch 
into it, and I thought a rubric would reassure them that the assignment was 
indeed to be taken at its word. At the same time, I didn’t know what to expect 
from them, and was charting new territory myself (see fig. 3). It did prove to be 
immensely useful when I evaluated the students’ projects and papers. 


Your projects will be evaluated according to the following criteria. In each case, the possible answers are: very 
well; well; somewhat; not very well; not at all. 

I. Objects 

1 . How well does the object express the dilemma it is meant to represent? 

2. How well does it incorporate materials from different sources that embody aspects of that dilemma in 
your life? 

3. How well does it present materials that have been processed in a way that is metaphorically suggestive? 

4. How well do the elements of the object come together into a whole? 

5. How well do the object and the dilemma it embodies relate to concepts we have studied in the course? 

II. Commentaries 

1. How well does the commentary explain the object and its representation of a dilemma? 

2. How well does it describe the processes used to create it, including any metaphors or metonymies? 

3. How well does the commentary draw connections between this particular dilemma and each of the read- 
ings that it integrates? 

4. How well does the commentary draw connections to your own life? 

5. How clearly and effectively is the commentary written? 

III. Grading 

Work that earns a grade in the A range will be marked “very well" in most cases above. “Well” translates to 
the B range; “Somewhat" to the C range, etc. This evaluation is an inexact science. The categories above are 
not equally weighted. Most important are: conceptual richness and clarity; connections to the readings; and 
creative thought. 

Fig. 3. Rubric evaluating the Museum of Dilemmas projects, Sarah Webster Goodwin, date unknown. 
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If I had at hand a rubric on assignments, to assess how well they serve 
students’ learning, I would check the boxes saying that this assignment clearly 
served the students’ engagement, reflection, and sense of intellectual commu- 
nity. Their papers lacked the polish and shape of the conventional, revised 
expository essay, but they were rich in some qualities that ranked high in this 
particular course: the ability to draw connections among the readings and 
concepts, and to see how the abstract issues we were considering manifest 
themselves in our actual lives. To use Blake’s phrase, the work they did, and 
my experience of it, were about as far from the “same dull round” of the 
classroom as I can imagine. The rubric, in this instance, emphasized creativity 
and the specific processes within which they would have considerable freedom. 
In doing this, and in assuring the students that the assignment was to be taken 
seriously in every detail, and that I would be looking closely at their work, the 
rubric seems to have freed them to do extraordinary things. 

Crucial to the success of this culminating assignment were several factors. 
The students had the opportunity to meet the artist, and Dario Robleto’s pas- 
sion, affability, genuine interest in them, and generosity in talking about his 
works and his life gave them the chance to feel like insiders in the museum 
and to understand the full scope of Roblcto’s ambition. His works are inher- 
ently powerful because of the materials he uses: he represents extremes of life 
and death and emotions. All of this helped to engage the students. But I also 
think an important factor was my own excited uneasiness about the assign- 
ment. It took me into new terrain, and I even found myself wishing I could do 
it myself. The project, I insisted to the students, was not a work of art and 
would not be evaluated as one; with its accompanying paper, it was both a 
visual and a verbal conceptual response to Robleto’s works and our readings. 
My own sense that this assignment was a risk and that a lot was at stake pro- 
vided an important part of the context for them. They were responding not 
only to Robleto and the readings, but to me, and their work taught me to 
take more seriously the ways that the dialogue between teacher and student 
informs their work. If there was a rubric that faculty could use to assess their 
assignments, here are some things that might be on it: 

1 . The topic is engaging for the faculty member as well as the 
students. 

2. The topic leads the students to take risks, to learn some new 
concepts and to synthesize familiar ones. 

3. The topic has a certain je ne sais quoi. And most importantly, 

4. The topic continues the dialogue between the teacher and 
the students in ways that are not fully scripted, channeled or 
contained. 
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Dialogue, Innocence and Experience 

Nurse’s Song 

When the voices of children are heard on the green 
And laughing is heard on the hill, 

My heart is at rest within my breast 
And everything else is still 

Then come home my children, the sun is gone down 

And the dews of night arise 

Gome come leave off play, and let us away 

Till the morning appears in the skies 

No no let us play, for it is yet day 
And we cannot go to sleep 
Besides in the sky, the little birds fly 
And the hills are all covcrd with sheep 

Well well go & play till the light fades away 
And then go home to bed 
The little ones leaped & shouted & laugh’d 
And all the hills ecchoed 


William Blake, Songs of Innocence (15) 

If we read this poem as a scene of instruction, as many of Blake’s Songs 
are, it seems to offer a benign view of teaching, a sort of “Summerhill” peda- 
gogy, child-centered, flexible, and communal. These children are a long way 
from putting pen to paper, and the Nurse is an even longer way from design- 
ing a rubric to describe her expectations of their learning. In the world of this 
poem, no one is worried about whether the village school will meet the state 
standards, nor whether these children will find jobs when they graduate or be 
able to perform adequately in them if they do. And yet I would argue that the 
poem — especially in the context of Blake’s work as a whole, which allows us to 
extend our interpretation pretty far — offers a real and feasible model for the 
classroom and a way to make room for sublime learning. 

Summarized briefly, the pedagogical reading of this poem might look like 
this: The Nurse possesses adult knowledge — of night, of time, of death, and 
of everything that they import into the world of the poem; she knows that the 
children need to learn these things. But when she instructs them, she also lis- 
tens to their response, and she learns something even more capacious from 
them: that they, and she, are part of larger natural cycles that can become a 
source of even greater knowledge and understanding. The poem presents two 
kinds of sublime knowledge: the terrifying one, and a reassuring one. Ending 
as it does on the idea of echo, in which Nurse, children, sheep and hills all 
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echo each other, and in which the poem’s own rhymes seem to extend into our 
reading space, this Nurse’s Song emphasizes reciprocity, dialogue, learning as a 
mutually reinforcing undertaking. With those qualities, our collective learning 
reaches to the farthest point: an understanding and acceptance of our human 
constraints, and a celebration of the ways that we can escape the “same dull 
round.” Blake presents an alternative scene of instruction in the “Nurse’s 
Song” from Experience, and in it he suggests that without those qualities of dia- 
logue and reciprocity, we have no resources to face human losses and our fear 
of death: 

Then come home my children, the sun is gone down 

And the dews of night arise 

Your spring & your day, are wasted in play 

And your winter and night in disguise. (23) 

This Nurse speaks essentially to herself, and from the children she hears 
“whisperings” rather than shouts of joy. The affect is all negative and inward- 
turning. I would venture to say that every teacher has moments like hers in the 
classroom, when we blame the students for their ignorance, silently envy them 
their youth, and distrust them in fundamental ways. We probably all carry 
both models of teaching within us, along with others. What we may not do, 
though, is consider what is at stake if we do not open ourselves to the kind of 
learning Blake is talking about. 1 ! 

We want our students not only to produce facsimiles of “good writing,” 
but to be “engaged.” This means that they are willing to work hard, and also 
that they care about the work: that they have positive affect — that they experi- 
ence joy. And perhaps also that they have opportunities to express negative 
affect, candidly, as part of the planned, anticipated learning experience. In 
these two poems, Blake suggests that our pedagogical thinking may start from 
one place (the poems share three lines), and then reach a fork: in one direction 
lie open and honest exchange, a transformation on both sides, shared pleasure, 
and a point that touches our deepest longing, our wish for our learning to be 
meaningful in the face of death. In the other direction lie “disguise,” a perpet- 
uation of roles and constraints, isolation, and an inability to touch together 
the core of what matters. 

In an actual classroom, we can’t approach sublime learning every Monday, 
Wednesday and Friday; if we did, it would become the “same dull round.” 

But I believe that my best teaching, and my students’ best learning, happens 
when I sense an openness in the students, and they in me, and we go to an 
edge and peer over it. 

Not only isn’t there a stable rubric for this kind of learning, it is singularly 
difficult to measure. When does measurement offer a clarifying and liberating 
constraint, a usefully defining boundary, and when does it prevent us from 
escaping the machinery of the quotidian? What does this escape look like, 
when it happens? We may not be able to define it in advance, precisely because 
it pushes beyond what is known. “Known” is a relative term: we feel our stu- 
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dents’ exhilaration when their conscious knowledge expands, when they are 
working at the edge of what they can articulate. It may involve paradox, 
taboo, conflict; it may elude logic and be expressed best by metaphor or 
metonymy; it may be tentative, contingent, fleeting. When it happens in our 
students, it has usually happened in us at about the same time: a risk, a discov- 
ery, something we didn’t know before. It may be more likely to happen if we 
invite it, by putting it on the rubric. 

Assessment 

How much of this dimension of the rubric in the classroom translates to 
the scene of programmatic assessment? The challenge here is one of scale: if 
the scene of instruction in “Nurse’s Song” has just one adult and a handful of 
children, can it be scaled up to the size of a department, a college, a universi- 
ty? I would propose that all of the conclusions above about rubrics in the 
classroom hold as well for the larger project of assessment. 

In brief: assessment rubrics that are based on clear and visible goals for 
student learning serve the democratic and liberating function of a liberal arts 
education by clarifying for all constituents what that education is about and 
how the student can expect to be transformed. Assessment rubrics have their 
limitations: they may encourage us to overlook qualities in student work that 
are not on the rubric but that we do or could value; we need to build in ways 
to counteract this. Dialogue is essential to the creation of rubrics and assess- 
ment processes: among faculty, between faculty and students, between disci- 
plines, even between campus constituencies and alumnae. The process of 
deciding what it matters to learn is ongoing and communal, and it responds to 
local contingencies and to shifts over time. 14 Like any rubric, those used for 
assessment are themselves subject to evaluation; this is part of the dialogue. 
And finally, rubrics may work best if they do not play it too safe. Assessments 
should and must bring faculty to reconsider some sacred cows. Should affect, 
for example, have a larger role in what we hope our students will learn? In 
addition to analytic acumen and proficiency in writing, should they learn to 
communicate visually? Do we have the courage to say that we aim for our stu- 
dents to be able to communicate effectively across racial identities, and to 
develop the pedagogies that make that possible? What are the most ambitious 
goals of a liberal arts education? Ambition has a place in every assessment 
rubric, and that may mean ferreting out unspoken assumptions and talking 
through strenuous differences. 

If we undertake the shaping of the assessment rubric in a creative spirit, 
and with an ear to the ground, we may find ourselves rethinking what we 
want our students to learn and how they are most likely to do that. As the new 
Valid Assessment of Learning in Undergraduate Education (VALUE) rubric 
for “Creative Thinking” says, the highest level of creative thinking yields work 
that “[ajetively seeks out and follows through on untested and potentially risky 
directions or approaches ...” and work that “ [t] ransforms ideas or solutions 
into entirely new forms” (Rhodes 27). We should ask this of our students, and 
of ourselves: if the rubric can help the teacher learn, as well as the student, 
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and if it can take both to the outer edge of our understanding, it will have a 
Prophetic or Poetic character. As Blake wrote, “Reason or the ratio of all we 
have already known is not the same that it shall be when we know more.” 


NOTES 

I All quotations from Blake are from the Erdman edition and use Blake’s 
eccentric spellings and punctuation. 

~ See Holmes, The Age of Wonder: How the Romantic Generation Discovered the 
Beauty and Terror of Science, 1 13 and passim. 

^ For an elaboration of this concept of sublime learning with an emphasis 
on the cognitive and affective processes the learner undergoes, see Donna 
Heiland’s essay in this volume. 

4 Blake wrote of a kind of dialectic between the “Prolific” and the 

“Devouring,” mutually dependent beings or forces of creativity and con- 
sumption: “Thus one portion of being, is the Prolific, the other, the 
Devouring: to the devourer it seems as if the producer was in his chains, 
but it is not so, he only takes portions of existence and fancies that the 
whole. But the Prolific would cease to be Prolific unless the Devourer as a 
sea received the excess of his delights” ( The Marriage of Heaven and Hell 39). 

^ According to Broad, “And thus was born what became the standard, tra- 
ditional, five-point rubric, by some version of which nearly every large- 
scale assessment of writing since 1961 has been strictly guided” (6). 

^ Or, as Grant Wiggins has put it: “A liberal arts assessment system has to be 
based on known, clear, public, non-arbitrary standards, and criteria. There 
is no conceivable way for the student to be empowered and to become a 
masterful liberal artist if the criteria and standards are not known in 
advance. The student is kept fundamentally off-balance, intellectually and 
morally, if the professor has a secret test and secret scoring criteria.” 

^ See Berry, ed., Dario Robleto: Alloy of Love for more information on the 
exhibition. 

0 

° To view some of his works, see the selected pieces on the D’Amelio Terras 
Gallery website. 

^ “If I feel physically as if the top of my head were taken off, I know that is 
poetry.” Reported by T. W. Higginson in an undated letter to his wife 
(Halio 41). 

Quoted by Paley (546), no edition given, but the text is from the King 
James version. 

II As Paley also notes (550), Blake illustrated this sentence on plate 9 of his 
illuminated Book of Job. For Paley, in this poem, printed in 1793, Blake is 
in some sense justifying the terror of the French Revolution. But it is 
clearly more than a simple allegory; it is a scene of sublime learning. 

1 ~ See also Huot and Williamson, “Rethinking Portfolios for Evaluating 

Writing: Issues of Assessment and Power,” and Ball, “Expanding the 
Dialogue on Culture as a Critical Component When Assessing Writing.” 
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My reading of these poems is indebted to Heather Glen’s classic book, 
where she notes the way the “Nurse’s Song” from Innocence is a dialogue in 
which both the children and the nurse listen to and echo each other (19- 
23). For more on how Blake’s Songs embodied critiques of contemporary 
pedagogies, see Richardson. 

Sec Morgaine’s “Developing Rubrics: Lessons Learned,” in Rhodes (1 1-14). 
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POSTHUMANIST MEASURES: ELEPHANTS, 
ASSESSMENT, AND THE RETURN OF CREATIVITY 

LUCINDA COLE 


I don’t pretend to know what art is, and I don’t know why people 
believe in it, but I know that they do. It’s a question of faith, 

maybe, needing to believe in something bigger than ourselves 

But one thing I’m fairly sure of: if something makes you ask the 
question, Is it art?, there’s a very good chance that it is. 

Mclamid, When Elephants Paint: The Quest of Two 
Russian Artists to Save the Elephants of Thailand (49) 


PEGGY MAKI CLAIMS THAT THE first phase of assessment is reaching 
some consensus about “collective expectations for student learning” (33), a 
daunting task in any environment, one especially tricky for the humanities, 
which seems to be suffering a collective identity crisis partially fueled by pres- 
ent economic realities. Even the idea that education is somehow transforma- 
tive is now being decried as a useless shibboleth of Arnoldian rhetoric, as one 
after another scholar asks us to overcome our revulsion to all things financial 
and consider how corporatism (Kirp), globalization and hypcrprofessionaliza- 
tion (Donoghue), social conservatism (Newfield), and misguided utilitarianism 
on the part of faculty themselves (Fish) have turned humanists into an endan- 
gered species. These institutional histories, following the model of Gerald 
Graff’s 1987 Professing Literature, reinforce the idea that humanities scholars 
have done a very poor job of defining and defending the value of the liberal 
arts. In his provocatively named The Last Professors: The Corporate University and 
the Fate of the Humanities, Frank Donoghue outlines two ways that humanities 
faculty can “resist their extinction”: first, while acknowledging our powerlcss- 
ness within “an increasingly global economy and an increasingly transactional 
economy of wealth,” we can challenge the idea that career-oriented educa- 
tional institutions will necessarily lead to secure jobs and a better quality of life 
for students, without at the same time suggesting that “we have something 
superior to offer” them (137). Second, having admitted our collective irrele- 
vance, we can study our own institutional histories, an effort that, ideally, will 
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thwart the dangerous tendency we have to “romanticize” our work, thereby 
allowing us to adopt a perspective on academic labor that could somehow 
help “forestall” our “own extinction” (138). 

Fish, Kirp, Donoghue, Newfield, and others have done much to articulate 
the contours of an educational industry whose business practices are increas- 
ingly modeled on those of a tarnished, post-2008 corporate America. They 
have also indirectly created a valuable context for understanding the wariness 
that many of our colleagues bring to assessment initiatives, initiatives that are 
easily and perhaps justifiably regarded as yet another symptom of an econom- 
ically induced standardization of knowledge and learning. Resistance to assess- 
ment among literary people is notorious and usually quite articulate. One of 
my colleagues, for example, refused to participate in an assessment survey I 
circulated, claiming that she has “reservations about the particular discourses 
[the survey] relies upon” and that she has “made a professional decision not to 
participate in them.” Another accused me, good-naturedly, of “authenticating 
a discourse which is to [her] repulsive.” While such direct responses bespeak a 
naked aversion to assessment and a refusal to cross some real or imagined 
political line, in my experience (first as department chair, then as interim 
director of college writing) resistance to assessment is more often passive, such 
as people “forgetting” to turn in writing samples for evaluation. Generally, I 
think, our colleagues are ambivalent about assessment, trying to cooperate 
with what are perceived as institutional imperatives while being skeptical 
about their source and value. 

Within literary studies, the general problem of whether or not assessment 
is an over-reaching tentacle of corporate America is further complicated by 
some of our own foundational values having been systematically dismantled, 
or at least thoroughly deconstructed, over the past forty years. “Creativity” 
was one of the first Enlightenment legacies to fall (or to take a pounding) 
under the pressures of 1 980s and 1 990s poststructuralism; a collection of 
essays entitled The Anti-Aesthetic, which I remember being thrilled to find on 
the shelves of my graduate school library in 1983, heralded and typified the 
kind of Marxian, postmodernist, feminist, and postcolonial analysis that would 
result in two generations of literary scholars defining their work as being radi- 
cally different from — and sometimes even opposed to — that of artists and cre- 
ative writers. Creativity, in short, became cast as a form of mystification from 
which the professional literary scholar was supposed to distinguish herself. It is 
therefore not surprising that, under new economic pressures, creativity’s famil- 
iar place within mission statements has now also come under suspicion. In Save 
the World on Tour Own Tune, Stanley Fish, touting the value of critical thinking, 
warns professors of literature that we should be doing two things and two 
things only: introducing students to new lines of inquiry, and teaching critical 
thinking and writing: “I’m all for moral, civic, and creative capacities,” he 
claims, “but I’m not sure that there is much I or anyone else can do as a 
teacher to develop them” (11). 

Fish’s words are intended to display a bracing dose of realism, but this cri- 
tique comes at an auspicious time in the development of “creativity” theory 
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and practice. “More than 50 colleges have decided to offer creative writing 
majors in recent years,” says one source, an increase coming at a time “when 
the number of English majors as a whole is decreasing” (Kaufman et al. 10). 

In the past twenty years, moreover, while we were reading and re-reading 
Freud, research on creativity became a multidisciplinary endeavor; drawing 
upon studies in medicine, psychology, education, neuroscience, and ethology, 1 
thinkers from other disciplines have been attempting to bring creativity into 
the realm of empirical analysis. Creativity has even given rise to its own assess- 
ment industry. As the authors of a recent book on creativity assessment point 
out, researchers of creativity, “believe that creativity is a natural candidate to 
supplement traditional measures of ability and achievement” (Kaufman et al. 
10); creativity was “one of six non-cognitive areas that Mayer (2001) recom- 
mended as being valuable candidates for new measures” and one of five “qual- 
ities singled out in a study of potential additional measures to the GRE” (11). 

In conjunction with an ongoing assessment project, I persuaded 1 1 of my 18 
full-time colleagues, few of whom had demonstrated any particular inclination 
towards the several assessment imperatives bearing down upon us, to complete 
an in-house survey entitled “Creativity Assessment.” Without being sure what 
specific attitudes would emerge, I expected to find patterns, and did. Those who 
did not balk at the language of assessment expressed predictable discomfort with 
the amorphous quality of the word “creativity,” which I had not defined. One 
after another attempted to think about it in ways that no longer smacked of a 
tainted romantic aesthetics. When asked whether or not they discuss creativity in 
their gateway courses to the major, for example, they responded with great care: 

“ [The word is] a bit too loaded, because we have to work against 
the common notion of creativity as ‘absence of structure.’” 

“I try to demystify the concept, if possible, and then revive it as a 
critical skill.” 

“Not that word . . . because [the students] don’t associate it with 
learning. Creativity for them often means a bolt from the blue. 

They think one creates ex nihilo — I’m going to hit it big! My job is 
to demonstrate that creativity comes out of a process.” 

A poet who also writes poststructuralist critical texts, perhaps traumatized by 
the culture wars of a previous generation, attached a warning: “I hope this 
survey is intended for responsible purposes and run by sane colleagues who 
know that creativity and intellectual ambition are not opposed.” His was one 
of many attempts to reinterpret the concept, to rescue “creativity” from its 
fuzzy and often pcdagogically ineffectual associations. 

Despite this trepidation about the word “creativity,” however, the question 
provoking the most consensus in my survey was this: “Do you believe a com- 
ponent of good scholarship is ‘creativity’?” Eighty percent “strongly believe” 
that it is. Most also “strongly believe” that readers of journals considered “cre- 
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ativity” in evaluating work for publication, and that some English majors are 
more “creative” that others. This consistency, along with some telling contra- 
dictions to be discussed shortly, suggests that whatever the economic, political, 
and epistemological problems surrounding assessment as a practice, such 
problems arc preceded and complicated for literary scholars by a different set 
of considerations, notably our collective and still-vexed relationship to the aes- 
thetic. Consequently, if we are ever to reach some consensus about “collective 
expectations for student learning,” we have to face — again — the aesthetic and 
its paradoxes, this time from a different vantage point than that offered by 
poststructuralist theory in the 1 980s. The purpose of this essay is to demon- 
strate how our tacit if sometimes embarrassed understanding of the contradic- 
tory nature of the aesthetic may affect our attitudes towards assessment, not to 
mention fears of our own collective “extinction.” In order to explore these 
relationships I begin in an unlikely place, with elephants who paint. 

Where Is Creativity? 

Most North Americans have probably seen evidence of popular culture’s 
fascination with elephant art. When Elephants Paint: The Quest of Two Russian 
Artists to Save the Elephants of Thailand was published in 2000, and in 2002 the 
television show 60 Minutes broadcasted “Elephant Painters,” Bob Simon’s seg- 
ment on the efforts of Russian performance artists Komar and Melamid. The 
video of Hong, a female pachyderm at the Maetaman Conservation Camp, 
remains one of YouTube’s most-watched films, at last count viewed over four 
million times (“Original Elephant Painting”). In this eight and a half-minute 
video, Hong painstakingly draws a por- 
trait of an elephant holding a flower (see 
fig. 1). Prints of the elephant paintings 
may be purchased through the website 
Exotic World Gifts, a portion of whose 
profits go back to the camps in Thailand. 

Hong and her fellow artists work in a 
nation which, according to its own 
tourism authority website, has been 
“turned upside down” (Lair). In 1850, 
there were an estimated 100,000 domesti- 
cated elephants and approximately 6 mil- 
lion people; today, there are 2,700 domes- 
ticated elephants and 64 million Thais. 

During the 1970s and 1980s, a growing 
foreign demand for teak and other hard- 
woods, coupled with internal conversion 
of forest land to agriculture, resulted in 
the rapid deforestation of Thailand; a 
1 989 ban on logging left most of the 
domesticated elephant population in 
Thailand without a habitat and without a 



Fig. 1. Elephant painting, Hong, date unknown; 
Reprinted with permission from ExoticWorldGifts.com. 
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legal job. The tourism industry — Thailand is visited by some 10 to 11 million 
tourists annually — helped supply both. It is estimated that some 70% of 
domesticated elephants are now engaged in some aspect of the tourist indus- 
try. Most of these elephants live in over 60 tourist camps, and an increasing 
number of them paint (Lair). 

These elephants and their mahouts raise a host of important and interre- 
lated issues about human/ animal relations and humans in relation to a 
changing natural world, among them the nature and value of creativity. 

Below the YouTube video of Hong are attached some eight thousand viewer 
comments which fall into three groups: those who think this painting is sign 
of elephant creativity, emotion, and/ or intelligence; those who think the 
video is fake; and those who believe the video demonstrates little except the 
elephant’s capacity to be manipulated or controlled. “The painting,” writes 
one observer, “shows no creativity on the animal’s part, as he’s just following 
the prompts to receive rewards” (Bogsnarth). Implicit in this latter view is a 
notion of the aesthetic as a spontaneous, unprovoked behavior which, espe- 
cially in the 1970s, structured, positively or negatively, most discussions of 
animals and art. Some readers might remember an elephant named Siri 
whose unprompted, abstract drawings became famous then. Siri’s trainer 
David Gucwa enlisted enlisted journalist James Ehmann, who sent copies of 
the drawings to scientists and artists for their interpretation. While a few peo- 
ple noted in these drawings signs of an artistic disposition — Willem de 
Kooning remarked, famously, “That’s a damned talented elephant” — most 
did not. Donald Griffin, the researcher who first discovered ccholocation in 
bats, said he found the drawings “intriguing” but because the drawings are 
free-form he “wouldn’t want to go very far in inferring consciousness”: “short 
of clear representation,” he writes, “I don’t know what the devil you could 
make of it” (qtd. in Gucwa and Ehmann 137). The relationship between con- 
sciousness and “art” reappears, forty years later, in consideration of Hong’s 
work but this time, the ability to draw rcpresentationally excludes the elephant 
painting from being regarded as “art.” Consider, for example, how 
Snopes.com — the great dcmystifier of Internet lore — responds to a viewer 
who, having seen the video of Hong painting an elephant, wants to know if 
the video is “true”: 

it is “true” in the sense that it represents the real phenomenon of 
elephants who have learned to paint, with the caveat that “paint- 
ing” in this sense means the animals outline and color specific 
drawings they’ve been taught to replicate (rather than abstractly 
making free-form portraits of whatever tickles their pachydcrmic 
fancies at the moment). (Mikkelson and Mikkelson) 

This author implies that only “free-form” painting could be taken as a sign of 
consciousness, as something other than mechanical bchavioralist prompting. 

Acknowledging similar contradictions in readings of human art, Mihaly 
Csikszentmihalyi, a psychologist at Claremont Graduate University, has long 
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argued that rather than asking what is creativity, we should be asking a different 
question: where is creativity? His shift in emphasis from individual works of art 
to their place within in a system is intended (like the work of Pierre Bourdieu) 
to provide a description of creativity within specific cultural and historical 
milieus: “what we call creative,” Csikszentmihalyi insists, “is never the result of 
individual action alone” but the product of three “main shaping forces”: 

a set of social institutions, or field , that selects from the variations 
produced by individuals those that are worth preserving; a stable 
cultural domain that will preserve and transmit the selected new 
ideas or forms to the following generations; and finally the individ- 
ual , who brings about some changes in the domain, a change that 
the field will consider to be creative. (“Implications” 325) 

Because creativity results from an interaction among the individual, the field, 
and the domain, “it is impossible,” he insists, to tell whether or not an object 
or idea is creative by simply looking at it. “Without a historical context, one 
lacks the reference points necessary to determine if the product is in fact an 
adaptive innovation” (326). In this view, the people who respond to Hong’s 
video on YouTube arc part of a phenomenon, and their judgments are pre- 
ceded and partially shaped by a network of actors, among them the mahouts 
or elephant trainers, visiting artists such as Komar and Melamid, patrons, 
agents, curators, museum directors, Internet consumers who buy the prints, 
and a host of animal rights activists who, in conjunction with artists like 
Komar and Melamid, have been persistent and effective in their attempts to 
represent elephant painting as the ultimate outsider art. Social agreement, 
writes Csikszentmihalyi, “is one of the constitutive aspects of creativity, with- 
out which the phenomenon would not exist” (327). 

Among the several advantages of Csikszentimihalyi’s systems approach is 
that removing creativity from individual consciousness allows us to treat it, as 
we do “reason,” as a specific cultural and historical phenomenon, one of 
whose functions has been to police the border between civilized and uncivi- 
lized, human and nonhuman. Artistic “consciousness” may have once stood as 
the last mark of humanness, but from a system theory’s perspective, elephant 
art — indeed, any art — is “creative” to the extent that it is regarded as a novel 
variation, one “instrumental in revising and enlarging the symbolic domain of 
the visual arts” (326). 2 Because consciousness is no longer a determinative 
issue, the debates about motivated and unmotivated behavior, interested and 
disinterested judgment have underwritten many versions of aesthetic theory 
that can now be recast as a function of the aesthetic system itself. If, for exam- 
ple, one wanted to argue that Thailand’s elephants exhibited in their painting 
a “natural” behavior which German scholars (or any reader of phenomenolo- 
gy) would recognize as Spiel, there is evidence. People who work with elephants 
report that pachyderms, both in captivity and the wild, pick up sticks and 
draw figures in the dirt, perhaps as a way of relieving boredom, perhaps as a 
positively pleasurable behavior (Gucwa and Ehmann 39-40). If one wanted to 
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argue, alternatively, that the elephants are simply following the requests of 
their trainers, that too would be possible to do. Hong, after all, paints slightly 
different versions of the same two or three pictures over and over again; for 
some of us (although not for Andy Warhol) such repetition signifies positive 
proof of mimicry. The important point here is that the problematization of 
the relationship between the artist and his or her product turns out to be one 
of the means through which the aesthetic ensures its own reproduction. 

Within systems theory, then, the question of creativity continues to have seri- 
ous — even ontological — implications, but those cannot be addressed primarily 
in terms of the relationship between an individual (human or elephant) and 
his or her work of art. 

The Disappearing Mahout 

Let me return to the literature classroom. My “Creativity Assessment” 
survey was one stage in a collaborative study of the only required course for 
our English major at the University of Southern Maine, ENG 245, “Intro- 
duction to Literary Studies.” This course was established in 1992 as part of a 
Fund for the Improvement of Postsecondary Education (FIPSE) grant to 
revise our undergraduate English major, a project that took about five years. 
We all have slightly different accounts of why ENG 245 was created but most 
would agree that a spate of hires in the late 1980s and early 1990s resulted in 
more theory-focused and theory-driven upper-division courses; we needed 
something that would introduce new English majors to different interpretive 
approaches. One third of the department teaches ENG 245, and it is the only 
literature course for which we have developed anything that resembles clear 
learning outcomes, although these are really more like guidelines: the course 
should emphasize close reading skills, introduce students to at least four differ- 
ent theoretical approaches, treat literature both before and after 1800, etc. 
Most people who teach ENG 245 do not list on their syllabi that manageable 
group of course goals that assessment officers, believing that students will not 
learn what they are not explicitly expected to learn, desire and promote. Since 
1992, we have met annually to discuss ENG 245, agreeing every year that it 
tries to do too much, admitting that students who define themselves as creative 
writers tend not to perform well, and wondering if we should institute an 
alternative to — or prerequisite for — the course. So far both the course and the 
major remain unchanged. 

Given its peculiar position within our curriculum, ENG 245 bears the 
burden of introducing students not only to an important set of skills but per- 
haps also to a particular set of values. My survey and related interviews were 
partly an effort to get faculty to articulate these. It quickly became apparent 
that, despite the lack of a shared set of systematically and collectively generat- 
ed learning outcomes for ENG 245, my department was more or less in agree- 
ment that the course should teach critical thinking — often described, Graff- 
like, as introducing students to a set of disciplinary codes — and should also 
encourage a particular sensibility, variously defined: 
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“I’m trying to get my students to have a type of experience, to 
see themselves as informed observers.” 

“I’m teaching a certain kind of open-mindedness.” 

“I want students to find pleasure in analysis.” 

“I want to see insight and engagement in students But I want 

them to work within the conventions, traditions, and conceptual 
frameworks of the course and the field, which they often see as a 
restriction on their innovation.” 

“I encourage them to become bricoleurs, to tinker with ideas, to 
make connections between seemingly unlike things.” 

“My goal is to get engagement; my courses are set up to get stu- 
dents to demonstrate they can put things together in an interest- 
ing way.” 

These statements use different metaphors for describing the ideal learning 
experience. The “values” are recognizable as the usual somewhat nebulous by- 
products of traditional liberal and aesthetic education, slightly revised to 
emphasize critical analysis: students are encouraged to put “things together in 
an interesting way” or to make “connections between seemingly unlike 
things.” Some terminology is drawn from diversity discourse (“open-minded- 
ness”), some from pedagogical theory (“engagement”), some from structural- 
ism and poststructuralism (“ bricoleurs ”) but the statements are alike in repre- 
senting this desired experience and accompanying subjectivity as the result of 
a complex process and cultural system. 

First, for most respondents, creativity is perceived as being what assessment 
professionals call “domain specific,” a quality that emerges — if it emerges at 
all — after lengthy exposure to a given field. 3 Thus whereas nobody who 
responded to my survey thought creativity was “very important” to their method 
of evaluation in introductory courses on literature, in 200-level courses, histori- 
cal period courses, and senior seminars, increasingly they treated creativity “as 
an important part of the grading criteria.” As one of my colleagues explains: 

Everything for me depends on the assumption that students are 
entering a discipline. They have to learn the codes. But any artisan 
knows you can be creative with codes; otherwise all chairs would 
look the same. The more experienced member of the discipline 
will recognize the codes at work, although he or she may not 
always be able to show this to someone who is not in the discipline. 

Creativity means recognizing the rules, recombining them, and transcending 
them; it is conditional upon having internalized rules apparent in work whose 
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value is partly dependent upon its difference from what came before. From 
this perspective, creativity is a kind of divergent thinking, domain-specific 
knowledge from which deviations may occur. 

Second, for many of the respondents, this idea that creativity is part of a 
domain-specific process is built on top of a self-contradictory, psychologizing 
model of the aesthetic; we are both spontaneously, naturally producing artists 
and, the products of careful training. One of the more telling moments in my 
interviews, then, was in response to a series of questions about whether or not 
creativity, however they defined it, “could be taught.” Four thought it could, 
but most noted their discomfort with the question: “We provide a context to 
channel people into doing creative things,” said one professor; “we don’t tell 
them what to do.” “Creativity can be modeled,” said another; “it can be fos- 
tered, not taught.” While it cannot “be taught, it can be unleashed.” 

Unleashed, fostered, channeled, modeled: all of these verbs presuppose a pre- 
existent something located, presumably, somewhere in the student, which we 
can bring “out.” 

In a paradoxical understanding which, I think, is peculiar to the aesthetic 
and to literary pedagogy, the opposition between work and play, training and 
insight, does not present itself as a problem to be resolved but as a formative 
dialectic. Once human consciousness is removed from the equation, the lines 
of this aesthetic tension stand out in bold relief. The Exotic World Gifts web- 
site includes a link to a list of questions and answers about elephant art, one of 
which is “How do you determine which kinds of work an elephant will 
paint?” 


We must see their work and whether they are good at lines or 
dots. Then it is discussed with their mahouts to see the possibili- 
ties of what they think are the elephant’s strongest style. The 
mahout must learn and understand that style of painting he 
chooses for the elephant and know how to draw this himself 
before he can train his elephant. Moreover, the mahouts need to 
learn how to mix color and how to select the appropriate color 
from books or professional artists. This makes the artwork much 
more developed. (“Frequently Asked Questions”) 

This description states in more direct language the presence of the creative 
field — what Csikszentmihalyi defines as “all those persons who can affect the 
structure of a domain” — and how part of our function is to “pass on the spe- 
cialized symbolic information to the next generation” (“Implications” 330). 
Significantly, however, it does so in ways that preserve a sense of elephant 
agency and individuality — note the emphasis upon being “good at lines or 
dots” — even while it stresses the role of the mahout in developing the ele- 
phants’ “natural” style. This characterization of the pedagogical relationship 
between the elephant and mahout strikes me as being more or less identical to 
that of colleagues who claim, as I would, that creativity can be “modeled” but 
not “taught.” Being preserved in both cases is the formative role of the profes- 
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sor/ mahout; being denied, however, is some Lockean notion of tabula rasa 
upon which the professor or mahout merely inscribes. 

This formative dialectic brings with it an ideal pedagogical structure: the 
disappearing preceptor. Vanishing pedagogical presence is demonstrated nice- 
ly in the YouTube video. Hong’s mahout appears only in the first minute; he 
walks out with the elephant, setting up the easel and box of brushes and 
paints. The hand of the mahout, however, is quickly removed from camera 
range, the audience compelled to focus upon Hong’s trunk. Through purpose- 
ful and practiced brush strokes, eventually the outline of Hong’s elephant 
drawing begins to appear. As it does, one hears the human responses her 
activity provokes: the tourist who (presumably) does the actual filming repeats 
several times “Oh . . . my . . . gosh,” an amazement replicated in many of the 
some 8,300 comments, where the most common words are “amazing!” or 
“awesome!” In both cases, the event of Hong’s painting produces moments of 
blocked speech, of disempowerment and awe, that function as markers of the 
sublime. Despite the illusion created by the video, however, the mahout is a 
full collaborator in Hong’s paintings: he draws the original painting, teaches 
the elephant to copy it, then, in the actual performance, chooses the elephant’s 
brushes and colors of paints. Among elephants who do more abstract work, 
this is how the painting proceeds: 

[The mahout] gives the brush to the elephant and maneuvers the 
trunk, now grasping the paint-soaked brush so that it meets the 
canvas. He then guides the trunk around the canvas in slow, cir- 
cular movements, giving the elephant a feel for the action desired. 

The hope is that, after some practice and verbal encouragement, 
the elephant will get the hang of it, and will begin to move the 
brush around independently — will begin, in other words, to paint. 
(Komar and Mclamid 44) 

The mahout, from this perspective, is as much an artist as the elephant, but to 
the extent he does his job successfully, his hand in the final product will be 
minimized, even disappear. 

At stake in the relationship between the elephant and the mahout is not 
only an affective relationship but a structural one, specific to the aesthetic 
domain. Structurally, the “disinterestedness” typical of aesthetic production 
and judgment is not only dependent, as others have argued, upon the erasure 
of economic motivation, but also upon the ability of the instructor to occlude 
the heavy hand of his or her training. Both of these principles are violated in 
the comments by Hong’s trainer, Noi Rakchang, who says about Hong’s art, 

“I would like my elephant to paint the Thai flag beautifully because many visi- 
tors and tour guides love it. Some guides told me to train her to draw the flag 
more beautifully. I will try my best training her to be more skilled” (“Frequently 
Asked Questions”). Stressing the local and commercial qualities of her paint- 
ing, his and the tourists’ role in determining what she draws, Noi Rakchang 
lays bare the pedagogical and the economic in ways that diminish elephant 
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agency and subordinate art to economic and political imperatives. The tension 
between motivated and unmotivated behavior, between work and play, so cen- 
tral to the aesthetic, threatens to collapse. While outside aesthetic discourse, this 
distinction matters little, in order for these paintings to be marketed as “art,” as 
something other than mere behavioralist scratching, the lines between natural 
ability and careful training must be blurred — blurred not because Hong is an 
elephant, but because she is an artist. Otherwise, the magic is gone. 

Roger Fouts, an animal ethnologist who lives with and studies a family of 
chimpanzees, asserts, “You don’t want to destroy an enjoyable natural behav- 
ior with rewards, which will turn it into work. What you’re after is the individ- 
ual’s creativity. In a reward situation, you end up with a trained-seal effect” 
(qtd. in Gucwa and Ehmann 153). Fouts’s categorical contrast between “enjoy- 
able natural behaviors” and the “trained seal effect” may put the issue too 
crudely for some professors of literature, but it is hard to deny a similar ten- 
sion in our pedagogy. Despite our theoretical disagreements and a history of 
institutional conflicts outlined over twenty years ago by Gerald Graff, many of 
us seem to have emerged with some similar attitudes about our function as 
teachers. Underwriting our pedagogy is not simply an attempt to distance our- 
selves from a mimetic model of teaching, but to do so in ways that are (con- 
sciously or unconsciously) enabling of the aesthetic. Creativity requires its aes- 
thetic opposite, imitation. Any aesthetic product — student essays on Moby Dick 
or elephant art — will be haunted by the possibility of copying or parroting, 
the “trained seal effect” by which any aesthetic endeavor can be undermined. 
Trepidation around the word “creativity” notwithstanding, still built into our 
evaluative judgments is a contrast between students who “merely” feed back 
codes and students who use them in more individualizing ways. Doubtless, 
awareness of how subtle this difference can be makes literary scholars espe- 
cially nervous around pedagogical initiatives that sometimes seem to do little 
except encourage students to learn to respond to a given set of cues, even as 
we know that teaching students to demonstrate a certain control of subject 
matter is an important part of our jobs. As a professor of literature admitted, 
“You have to give good grades to students who can just parrot things back.” In 
human pedagogy, “teaching to the test” — “What do I need to know, 
teacher?” — emerges as the evil twin of genuine instruction. In our reptilian 
aesthetic brains — in the part that allows us to say, if only to ourselves, “Man, 
this poem sucks” — we sense that the “A’ student needs the “B” student, in 
much the same way that Mozart needed Salieri, and that some humans, in 
order to be reassured of their own species exceptionalism, need elephant “art.” 

Askesis , Motivation, and Assessment 

In using elephants to talk about creativity, literature, and assessment, I do 
not mean to suggest that faculty in my department bear a merely allegorical 
relationship to Thailand’s mahouts, and their elephants to our students. 

Instead this juxtaposition between elephant camps and literature classrooms is 
intended to emphasize two key points relevant, I think, to the contemporary 
discussions of assessment. First, humans and elephants alike exist in a global 


163 



LITERARY STUDY, MEASUREMENT, AND THE SUBLIME: DISCIPLINARY ASSESSMENT 


economy in which creativity, far from being an immaterial mystification, is a 
mystification with power and even market value. Second, central to that mar- 
ket value is a pedagogical structure in which the hand of the instructor is nec- 
essarily minimized. By virtue of her seemingly natural performance, after all, 
Hong becomes endowed with individuality, a distinct self. Even if this self is a 
mere “reality effect,” that reality effect has significant economic consequences. 
The elephant and her mahout no longer roam the streets. And Hong’s paint- 
ings sell, so from the perspective of the Mahetaman Elephant Art and 
Conservation Camp, the pay-off for intense elephant “engagement” is an 
influx of tourist dollars which, in turn, are used to support other projects: a 
milk bank in Thailand, an elephant reproduction facility, a new training 
school for future mahouts. While the aesthetic involves a kind of labor, and 
while that labor is doubtless immersed in business practices over which the ele- 
phants have little control, painting pictures, in the grand scheme of things, is 
qualitatively different from hauling logs. So is teaching literature. 

These two economies can be distinguished primarily by the perceived dif- 
ference between intrinsic and extrinsic motivation. Many literary professionals 
bring to their teaching a particular kind of self-understanding: that teaching is 
not merely a job, but a mode of askesis typical of the aesthetic. This line of 
ethico-aesthetic thinking emerges pretty clearly in the later work of Foucault. 

In Care of the Self and in various seminars and interviews, Foucault uses the 
term “aesthetic” not in its narrow meaning as a separate sphere of existence 
occupied by painters, poets, and philosophers whose ideas bear an elliptical 
relation to material reality, but instead to signify modes of askesis or “technolo- 
gies of the self” through which we come to see ourselves as ethical subjects. 

Ian Hunter, drawing on Foucault, similarly defines the aesthetic as a “distinc- 
tive way of actually conducting one’s life — as a self-supporting ensemble of 
techniques and practices for problematizing conduct and events and bringing 
oneself into being as the subject of an aesthetic existence” (348). Theorized in 
this way, Hunter continues, the aesthetic domain may be relocated from a his- 
tory of ideas (and ideology) to a history of “practices of the self”: 

that is, a history of the means by which individuals have come to 
form themselves as the subjects of various kinds of experience 
and action and to endow their lives with particular kinds of sig- 
nificance and shape. These are the practices that Weber identifies 
with Lebensfiihrung or the conduct of life and that Foucault 
describes as ethical technologies. (359) 

Such practices are performative and may or may not pertain directly to an art 
object. While Fish, for example, does not address directly address askesis in Save 
the World On Tour Own Tune , it is possible to locate traces of this ethico-aestheti- 
cism in his argument for the value of critical analysis. “A sustained inquiry 
into the truth of a matter,” Fish writes, “is an almost athletic experience; it 
may exhaust you, but it also improves you” (40). But motivation for this “expe- 
rience,” he insists, is necessarily intrinsic: “If you are committed to an enter- 
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prise and have internalized its values, you don’t spend much time asking ques- 
tions like ‘what is this good for?’ You have already answered that question by 
sticking with the job: it’s good because it’s what you like to do” (59). Fish iden- 
tifies his refusal to embrace a particular political or ethical position in his class- 
rooms as “minoritarian,” which it may now be, but in departments of English, 
the idea that teaching is a mode of askesis has retained its power. ^ “Teaching,” 
says one of my own colleagues, “is a kind of performance art. It takes a great 
deal of creativity to be a good teacher.” “Creativity” is offered not simply as a 
criteria descriptor of a quality we hope to find in a student paper; instead, it is 
something many of us, even those critical of traditional aesthetic discourse, 
need to believe that we are, and do. In this regard, for many professors teach- 
ing and learning are partially driven by knowledges intrinsic to the field and 
partially — though we less often discuss this aspect — by intrinsic motivation, by 
motives that are perceived (correctly or incorrectly) as coming from “within” 
(Nickerson 412; Collins and Amabile 300). 

Our historical relationship to aesthetic discourse is still applicable in add- 
ressing faculty resistance to assessment. Members of departments of English 
are caught between complex disciplinary imperatives and the demands of a 
renewed corporativism, with its “data-driven decision-making,” “student-cen- 
tered learning,” and economically-driven desire to enlist public funding and 
trust. Faculty have been called upon to help foster “learning environments” 
that, in the words of the widely influential Kellogg Commission on the Future 
of State and Land-Grant Universities, “meet the civic ends of public higher 
education by preparing students to lead and participate in a democratic socie- 
ty” (39). Within this context, assessment plays a critical but highly problematic 
role. On the one hand, the Kellogg Commission pays homage to the aesthetic 
by borrowing the language of “creativity”: “Good teaching is a form of cre- 
ativity that links discovery with integration and application,” a “process ... of 
creating contexts in which students, whether young or old, can grow into the 
fullness of their uniqueness as human beings” (40). On the other hand, creating 
such “contexts” turns out to be dependent upon a third term having no imme- 
diate relationship to either students or teachers, but primarily to practices of 
assessment: “To provide the best possible basis for our efforts,” insist these uni- 
versity presidents and chancellors, “we need to encourage research on the 
learning process itself, with the goal of creating a science and a methodology to 
discover how we can most effectively present knowledge” (32). In linking assess- 
ment to “good teaching,” then, the report smuggles in the language of creativi- 
ty but uses it in ways that strip faculty of the intrinsic motivation thought cen- 
tral to the aesthetic. Instead of being regarded as an equal partner in a form of 
parallel play, or instead of being regarded as one partner in a temporary 
apprenticeship, within the “student-centered university,” faculty are expected to 
serve as one unit in a twenty-four hour delivery system “committed to excel- 
lence in teaching and to meeting the legitimate needs of learners, wherever 
they are, whatever they need, whenever they need it” (12). 

Much assessment literature simply replicates the goals, values, and prob- 
lems of the Kellogg Commission, thereby helping to create the conditions for 
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widespread resistance to assessment. In a “learning-centered institution,” writes 
Peggy Maki, faculty “become designers of environments and tasks that foster 
student discovery and the construction of meaning rather than predominantly 
transmitters of knowledge” (10). Her division of faculty into two groups — 
“designers” and “transmitters” — is, consciously or unconsciously, politically 
coercive; it compels faculty to choose between being regarded as “creative” 
teachers who care about their students, or as sub-professional people who self- 
ishly “imitate” teaching by simply “transmitting” knowledge. Especially for lit- 
erature faculty who a) teach writing and b) are trained to negotiate the complex 
demands of aesthetic discourse, the spitting forth of received knowledge has 
rarely been a viable or desirable pedagogical stance. For many of us, the 
“learning-centered institution” promoted by administrators and assessment offi- 
cers therefore either seems beside the point or a not-so-subtle way of coercing 
faculty into assessment initiatives whose relevance has not always been suffi- 
ciently established. Given our historical relationship to the aesthetic, literature 
faculty are, for better or worse, structurally positioned to recognize that the 
very definition of the “student-centered university” represents a radically sim- 
plified and perhaps unnecessarily divisive notion of the pedagogical task. 

“Confronting the issue of faculty motivation is certainly the most difficult 
task of assessment,” as Maki acknowledges in Assessing for Learning: Building a 
Sustainable Commitment Across the Institution. She writes that “directing professional 
energy and curiosity into what and how students learn ... is an essential process 
in a learning-centered institution” (1 1). Maki’s notion of “directing” profession- 
al energy may, however, be part of the problem, though she comes by it honest- 
ly: one of the first principles of the Kellogg Report is that “engagement” takes 
strong leaders. Leadership “to create an engagement agenda is crucial,” we are 
told. “Engagement will not develop by itself, and it will not be led by the faint 
of heart” (25). In contrast to these military metaphors, Csikszentmihalyi 
reminds us that fostering creativity is a delicate task dependent upon a curiosity 
that needs to be “cultivated.” “Each of us,” he writes, 

is born with two contradictory sets of instructions: a conservative 
tendency, made up of instincts for self-preservation, self-aggran- 
dizing, and saving energy, and an expansive tendency made up of 
instincts for exploring, for enjoying novelty and risk — the curiosity 
that leads to creativity belongs to this set. We need both of these 
programs. But whereas the first tendency requires little encour- 
agement or support from outside to motivate behavior, the second 
can wilt if not cultivated, (qtd. in Nickerson 411) 

If too few opportunities for curiosity are available, he continues, or if “too many 
obstacles are placed in the way of risk and exploration, the motivation to engage 
in creative behavior is readily extinguished” (qtd. in Nickerson 41 1). One won- 
ders if the ideal way to cultivate faculty interest in assessment is by enlisting a bat- 
tery of “strong leaders” in an environment widely perceived as being unsafe. 
Surrounded by administrators and new mission statements and struggling assess- 
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ment officers waving the banner of “student-centered learning,” faculty who 
question assessment efforts are too often cast as the Phantom of Student Learning 
Past, a creature haunting the halls of academia, wielding forty-year-old lecture notes. 

David Kirp argues in Shakespeare, Einstein, and. the Bottom Line that today’s lib- 
eral arts professors are much “like Swiss watchmakers, offering what is widely 
regarded as a luxury item to a shrinking clientele,” treating the “value” of our 
subject as “self-evident” (258-59). A more sympathetic rendering might be that 
professors of literature have not been the hapless and ineffectual aesthetes that 
they have been painted, but accidental victims of their pedagogical success. 
Having intentionally obscured the many techniques through which generation 
after generation of literary students has been trained to read and write about lit- 
erature, partially for the sake of preserving a studied disinterestedness, professors 
are reluctant to discuss pedagogical strategy and have no ready protocol for 
demonstrating the nature of what they do. Nor is there much intrinsic incentive 
to develop a protocol, given that the very act of demonstration threatens to col- 
lapse that still-relevant distinction between “motivated” and “unmotivated” 
thought, and action. Finally, many literature faculty are, or imagine themselves 
to be, driven primarily by an “intrinsic” motivation that includes the desire for 
autonomy. Even if the assumption of autonomy is a collective self-delusion, it 
functions as an enabling fiction, one allowing us to experience work as creative, 
our teaching as the closest thing to non-alienated labor one can expect. 

Coda: Can These Professors Be Saved? 

When Melamid is asked if the elephants really know what they are doing 
when they create paintings, if they understand they are making art, he shrugs. 

We don’t know what we are doing when we make art, so why 
should they? We think we understand, we use words like 
“because” — a painting is good because the artists had a certain 
feeling or idea, because he somehow turned that idea into an 
image — but really we have very little idea what we mean by that. 

We use words like genius, brilliant, spirit, but we don’t know what 
these things mean, either. (49) 

He is right. Despite centuries of scholarship on the relationship between indi- 
vidual psychology and art, for example, on “work” and “play,” no one has ever 
been able to determine in empirical ways when a (human or non-human) ani- 
mal “enjoys.” Studies proliferate, including some very recent ones of trained 
seals whose heart rates, measured on “naive” or self-motivated dives, are then 
compared to prompted ones; presumably, the faster heart rates on the former 
indicate a different affect, perhaps one more closely associated with pleasure 
(Jobsis et al. 3884). Animal right activists continue to debate whether it is cruel 
to have draft horses haul three times their weight in “pulls,” or whether this is a 
naturally occurring impulse they “enjoy.” Neurobiologists equate bird song with 
speech, and thus with learning, while musicologists treat it as a mode of natural 
self-expression (Kaplan and Rogers 4-6). The lines between motivated and 
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unmotivated behavior in elephants, whales, birds, seals, and chimpanzees, then, 
are at least as blurred as they are in many discussions of humans. Discussions 
of “creativity” in both cases arc similarly structured through an emphasis upon 
individual psychology, the aesthetic product, and a not unproblematic opposi- 
tion between intrinsic and extrinsic motivation. 

This division between intrinsic and extrinsic motivation is, of course, often 
more strategic than real, obscuring, as it does, the complex social and econom- 
ic interdependencies through which we — humans and non-human animals — 
live and function. As Thailand’s painting elephants demonstrate, to work in a 
creative economy does not mean one has entirely shed one’s chains. Maine, 
for example, is much like Thailand, a fragile tourist economy attempting to 
recover from a radically decimated logging industry. Its public universities 
have been rebranding themselves as “student centered” for several years; the 
University of Southern Maine (USM) runs a fleet of buses painted with fresh- 
faced “life-long learners” and a new motto, “Education Your Way.” As the 
resemblance between this ad campaign and that of an international fast food 
chain suggests, Maine’s public university system is taking seriously the Kellogg 
Commission’s injunction to serve students “wherever they are, whatever they 
need, whenever they need it” (12). One of our more notable successes in this 
regard is a low-residency program in creative writing, the Stonecoast MFA 
which, in USM’s new strategic plan, is singled out as a model for the rest of us 
to follow; the director of the program, a nationally known poet, tells me that an 
MFA in creative writing was recently described on NPR as “the new MBA.” 

While it is hard not be a little cynical about this state of affairs, the eco- 
nomic success of MFA programs here and elsewhere lead me to think David 
Kirp overstates the case in his claim that the liberal arts have “lost much of 
their clientele ... as career-minded undergraduates have shifted their alle- 
giances to the practical arts” (258-59). Instead, the liberal arts, and especially 
those seeming to offer a skill set imagined as definitional of the “human,” are 
being repackaged by the same folks who brought us assessment and “value- 
added” education. Such circumstances call for an environmentally sensitive 
adaptation on our part. “Whether we like it or not,” insists Csikszentmihalyi, 
“our species has become dependent on creativity” (“Implications” 318). I have 
been advocating that we acknowledge the continued relevance of “creativity” 
in any number of discourses, including, as we have seen, environmental ones; 
for Thailand’s elephants, the performance of creativity has become a matter 
of survival. While emphasizing our roles as teachers of critical analysis, as 
Fish suggests, is one way for literary professionals to adapt to the new culture 
of measurement, another may be to reclaim aesthetic discourse from the 
social sciences. It’s worth a try. As Slavoj Zizek, echoing Lacan, writes in In 
Defense of Lost Causes, sometimes, rather than “running after ‘objective’ truth,” 
it is better to hold “onto the truth about the position from which one speaks” (3). 


168 



POSTHUMANIST MEASURES: ELEPHANTS, ASSESSMENT, AND THE RETURN OF CREATIVITY 


NOTES 

1 A very good bibliography on creativity research can be found in Kaufman et al. 

^ Admittedly, Csikszentmihalyi does not make this point about animal art. He 
writes, “To be human is to be creative” ( Creativity 318), but following the 
logic of Iris argument, it does not follow that to be creative is to be human. 

* ?l On general versus domain-specific approaches to creativity assessment, 
see Csikszentmihalyi, Creativity 23-51 and “Implications” 314-16. 

^ While Fish promotes the values of disinterestedness, it is worth pointing 
out that any technology of the self could be regarded as a mode of askesis, 
including those allied with a particular identity or politically motivated 
subject position. 
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ASSESSMENT IN LITERARY EDUCATION 

CHARLES ALTIERI 


I THINK VERY FEW OF US would not welcome more cogent attention to 
processes of assessment. It is obviously salutary to imagine teaching as a con- 
tract made with students to develop certain skills, powers, and bodies of 
knowledge. And it is apparent now that faculty members should make public 
on a website their specific goals for each of the classes they teach. We also 
have to realize that personnel committees should do even more vigilant jobs in 
judging how individual instructors live up to their principles and prove effec- 
tive in the classroom. More important, we now have the technology to enable 
faculty to post representative papers (with the grader’s comments but the 
author’s names blacked out) indicating what kind of work merits each grade 
from them. These postings can then also serve as indices of student growth if 
the faculty member indicates where they come in the course of the semester. 

Yet the more avid proponents of assessment are not content to raise 
awareness and develop more careful modes of scrutiny. They point out that 
these personnel committees arc bound to subjective judgments, usually 
reduced to attending to student evaluations and, at best, one class visit. Such 
committees rarely examine how teachers grade or seek measures that will tell 
how much the faculty member has actually improved the skills that students 
bring to the course in the first place. And they almost never rely on objective 
standards that might provide a comparative measure of teaching effectiveness 
that can be used also to assess entire programs, especially for those who are 
not within the academy. Without such objective standards even the prolifera- 
tion of information about faculty work might make it more rather than less 
difficult to make these comparisons. 

We have to respond to these new demands in two ways. We have to ask 
theoretical questions about whether such objective standards are feasible and 
helpful in the various disciplinary frameworks the university cultivates. And 
even if we think such objective standards feasible, we have to ask practically 
whether they are worth the labor and expense involved. Perhaps it might be 
wiser to spend the money creating smaller classes so faculty can pay more 
attention to the student efforts that are being assessed. 

At first glance it should be theoretically possible to establish objective 
measures for degrees of success in teaching. After all the goals of teaching are 
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implicit in the discipline; they need not be imposed by some foreign theoreti- 
cal apparatus bound to a specific ideology The teaching of writing should 
produce improvements in the students’ uses of grammar, in their ability to for- 
mulate and criticize arguments, and in their facility with the language. If 
someone cannot show improvements in these domains it should be clear that 
he or she is not doing the contracted job or doing it in a much less than opti- 
mal way. Similarly if someone is teaching any kind of course that stresses 
reading, students should manifest improvement in their ability to summarize 
arguments or plots and in their ability to characterize how particular texts 
respond to the struggles and to the conventions that comprise relevant histori- 
cal contexts. 

Indeed arguments like these have been persuasive for many influential 
and impressive leaders in the field of composition studies . 1 But then we 
encounter a disturbing problem: why have so few professors who emphasize 
literary criticism joined their colleagues? Do they constitute a privileged rem- 
nant too lazy or too self-satisfied to acknowledge that something has to be 
done to develop public confidence that society’s investments in higher educa- 
tion are worth the expenditure? Or does their involvement in the decidedly 
unfashionable study of literary texts make them unreasonably suspicious of 
anything that seeks objective knowledge and invites the heathen masses to 
make assessments about the high mysteries that it is their calling to defend? 

I think there is another cause for their resistance to strong claims for 
assessment. There is first a justified suspicion that assessment and authority do 
not go well together. Everyone should be as transparent as possible about 
classroom goals and how he or she measures success. But this is a far cry from 
agreeing that it is even possible for public bodies removed from the classroom 
to develop measures that would have to apply uniformly to quite diverse cases. 
And these measures would impose another level on the educational process. It 
would be difficult not to teach to the measures once they are in place — we 
have learned this much from “No Child Left Behind.” We would enter a situa- 
tion where we have to trust in some abstract indicator of teaching success that 
is likely to be insensitive to what makes us commit to teaching in the first 
place. And the promise of comparing classes or institutions under one rubric 
is likely also to collapse all sorts of differences that make institutions and indi- 
vidual teachers attractive in the first place. 

These suspicions are considerably deepened when we turn to theoretical 
frameworks that have traditionally been central to the teaching of literary 
texts, and have therefore shaped many of the expectations for professors of 
what is involved in the successful teaching of these materials. We will see that 
if we are to teach literary texts as aesthetic objects (at least in part) we also 
have to emphasize writing about these texts. And this kind of writing cannot 
dwell on these texts as objects of knowledge, or even objects that solicit knowl- 
edge about the culture. Rather such writing is best conceived as showing how 
one can participate imaginatively and affectively in the experiences that are 
promising to modify and reward our sensibilities as they strive to identify pro- 
visionally with particular struggles and particular modes of attention. These 
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provisional identifications typically do not offer themselves as vehicles by 
which we develop knowledge and transform fleeting experience into stable 
generalizations. Instead they ask to be evaluated simply by how the writing 
articulates the student’s capacities for responsiveness to particular qualities in 
the work. 

There are many elements of this perspective on literary education that I 
will have to unpack and clarify. So I will offer an account borrowed largely from 
Hegel about what is involved in getting students to respond to the aesthetic 
dimension of what we teach. 2 Hegel will also indicate why the project of culti- 
vating powers of aesthetic judgment can serve as a paradigm case for difficulties 
that arise on various levels in any educational enterprise in which writing is an 
important component intended to demonstrate how the students develop their 
own capacities for various kinds of judgments. Being clear about the aesthetic 
also entails recognizing the difficulty of postulating determinate practical out- 
comes when one wants not only to communicate a body of material but also to 
model a standard for how to modify our sense of value through that reading. 

For many of us, teaching literature begins with the effort to communicate how 
that discipline differs — for better and for worse — from the kinds of disciplines 
devoted to communicating directly usable knowledge. In teaching aesthetic val- 
ues we have to emphasize considering states and values like attentiveness, inten- 
sity, force, and complexity that are extremely difficult for anyone to measure. 

But, I hasten to add, the difficulty is precisely the reason why this kind of teach- 
ing has to be encouraged, because it aims to articulate values that affect main- 
stream life while being inaccessible to its dominant practical languages. 

Now I face the challenge of being persuasive about how the goals for 
teaching literary appreciation can be clear and at the same time indicate how 
difficult it would be to translate that clarity into the practical testing of pro- 
posed outcomes. Invoking Hegelian abstraction will seem to many a strange 
means of confronting that challenge. But I think his abstractness makes it pos- 
sible to present the case in its most elemental form. For the abstraction is not 
so much an evasion of concrete particulars as an attempt to capture conceptu- 
ally what is involved in valuing those particulars. 

My specific focus will be elaborating three “common ideas of art” 3 that 
Hegel puts at the center of his “Introduction” to his Aesthetics'. 

1 . The work of art is no natural product; it is brought about by 
human activity; 

2. It is essentially made for man’s apprehension, and in particular is 
drawn more or less from the sensuous field for apprehension by the 
senses; 

3. It has an end and aim in itself. (25) 

These propositions, especially the third, are not uncontroversial. But minimal- 
ly they clearly define the beliefs involved if we are both to distinguish art from 
other modes of activity and to insist on the degree to which teaching literature 
is training in responsiveness to art objects. 
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Let me not be so happy in my abstraction that I dismiss examples. 
Examples measure the applicability of abstractions while abstractions articu- 
late what might be possible to exemplify. For brevity’s sake I am going to 
emphasize two one-line speeches by Cleopatra in Shakespeare’s Antony and, 
Cleopatra, one from the beginning of the play and one from 3.13 just after a 
Roman envoy offers promises to Cleopatra. The first passage comprises 
Cleopatra’s first words in the play, “if it be love indeed, tell me how much” 
(1.1.14); the second is her response to Antony’s shame and jealousy as he abus- 
es Caesar’s messenger, “Not know me yet” (3.13.160). Both passages invite his- 
torical speculation about how Cleopatra dealt with imperial power. But I sus- 
pect that most university teachers of this play still focus their attention on how 
Shakespeare interprets and embodies the psychology of Cleopatra’s dealing 
with imperial power — hence the importance of Hegel’s first and second 
claims. 

Notice all that Cleopatra’s first speech accomplishes. This speech occurs 
just after the play has given the typical Roman soldier’s view that their com- 
mander has become “a strumpet’s fool” (1.1.13) as he devotes himself to 
becoming “the bellows and the fan / to cool a gypsy’s lust” (1.1.9-10). But 
these words are the not the expression of a strumpet, and not even an exercise 
in lust. Cleopatra defines herself primarily as someone who mocks calculation, 
demands infinite attention, and possesses a complex mind capable of only 
half-believing in the games that the monarchs are playing about love. Speaking 
quasi-seriously in this way just is an intense form of erotic love, primarily 
because only that kind of love can occupy the space between the practical and 
something like an absolute domain where people contour themselves to intri- 
cate exchanges involving different degrees of belief and possibility. And the 
love is not merely narcissistic. Cleopatra taunts Antony in a way that reflects 
her understanding of his proclivity to shame while obdurately refusing to let 
him give himself over to something that in her eyes is not his best self. She 
uses the language of quantity to remind him that this is not the language that 
could even approximate what they share. This sharing takes performing the 
love, not trying to measure it. 

Much more can be said about Cleopatra’s opening speech. But we have 
seen enough to appreciate how Hegel’s first claim is carefully worded to pro- 
duce several important corollaries. Art is not a natural product because there 
is no rule of nature that explains its production or its use. There is no discov- 
ering in nature a Cleopatra or prototype for her, and there is no deciding from 
nature what the dramatic reality of Cleopatra has to be. The Roman soldiers 
try that route. We do not recognize Cleopatra as a natural creature but as in 
every word and image Shakespeare’s product. It is Shakespeare who creates 
the possibility of a character whom we have to take as providing a model for 
nature. And that possibility is not quite a matter of a single creative idea. 
Rather it stems from a continuous sense of invention that establishes endless 
surprises as the character in effect learns its own possibilities for establishing 
an identity. The authorial activity becomes a fundamental internal feature of 
what we take to be the power of the characters to invent themselves. 
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As Hegel develops this first claim he recognizes how dependent the aes- 
thetic domain is on what can be known objectively (and assessed as knowl- 
edge). There is a purely technical side of making that should become an 
object of study: “Skill in technique is not developed by any inspiration, but 
only by reflection, industry, and practice.” Without such skill one cannot be 
expected to “master” intractable “external material” (27). And where there is 
technique there are particular histories of its development and its distinct uses 
that are nourished by comparison with other skills and genres. But this 
acknowledgment also makes manifest the significance of what cannot be quite 
grasped as knowledge but has to be experienced as authorial intelligence pur- 
posefully putting qualities to work for quite specific imaginative purposes. 

This sense of pervasive intelligence need not accompany the awareness 
that the particular is a product of human labor rather than of nature. In fact 
there is here a contrast quite useful for defining the distinctive role of art in 
most societies. Many products tend to take on something like a second 
nature — in the sense that we treat them primarily as aspects of a cultural 
landscape that we know how to use. We simply accept them as already catego- 
rized. For example, I think it is a crucial fact that most of us do not worry 
about how light bulbs are invented or produced. They are objects ready to 
hand. We concentrate on what they are useful for rather than on the manner 
of their invention or substance. Art works have a very different status. 
Cleopatra is not a determinate object in our world with clear uses. Rather she 
becomes a living particular capable of changing and growing to the extent 
that we can gain further appreciation of what the making process establishes 
as her distinctive traits. Rather than assume we know what to do with her, we 
have to attend to what our attention might reveal about her and about our 
capacities to respond adequately to the particular emotions she presents. 

In choosing Cleopatra as my example I probably cheat a little by making 
it easier to illustrate what Hegel means in his second claim about the sensu- 
ousness of art. She is nothing if not sensuous. Even her claim on Antony to 
tell her how much he loves her is ultimately a bid to have all her senses active 
and self-aware. But the sharpness of the example provides a timely contrast to 
a social and critical order obsessed with acts of interpretation eager to trans- 
form every sensuous detail into an allegorical meaning. With Cleopatra 
Shakespeare established an imaginative object insistent on its sensuous being, 
and insistent too then on the importance of foregrounding the particularity of 
the work. Shakespeare seems less interested in what Cleopatra might stand for 
than in how her ways of standing elicit telling reactions from the other charac- 
ters and so contribute to establishing a dense singular world. 

The crucial point of assessing assessment is how Cleopatra’s sensuousness 
manifests the importance of distinguishing sharply the particular interrelations 
that a work offers from anything for which we can claim knowledge. To claim 
knowledge one has to show how the particular is an instance of a class or cat- 
egory with determinable relations to other elements within the category. But 
here all of Shakespeare’s skills seem devoted to showing how the imagination 
through the particular can take up residence in the sensuous. It is less impor- 
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tant to formulate ideas about Cleopatra than for students to give evidence that 
they can feel their way into her distinctive ways of processing experience. 
Indeed that is why there is an immense gulf between formulating why she says 
“if it be love indeed, tell me how much,” and identifying with this character 
who can at once make her lover ashamed and stimulated so that she can take 
her pleasure in her own strategies. 

Making the case for these values embedded in particularity requires Hegel 
to distinguish between levels of sensuousness. On one level the activity of mak- 
ing brings out signifying capacities in the sensuous material. Music orders sound 
and elaborates rhythms; literature awakens us to the capacities of language to 
become articulate — both semantically and sonically. But if we deal only with 
this level of sensuality we risk connoisseurship on the one hand and the cult of 
feeling or sheer reader response on the other. We can collapse the work into the 
expertise visible within the medium or we can collapse all sensuousness into a 
focus on how the work makes us feel. However, this is insufficient to what the 
human activity within the work can produce. I risk Hegelian language to 
demonstrate this because his statement is so powerful in defining a more capa- 
cious version of the sensuousness that calls upon our full imaginative energies: 

Of course science can start from the sensuous in its individuality 
and possess an idea of how this individual thing comes to be 
there in its individual color, shape, size, etc. Yet in this case the 
isolated sensuous thing has no further bearing on the spirit, inas- 
much as intelligence goes straight for the universal, the law, the 
thought and concept of the object On the contrary the sensu- 
ousness in the work of art is itself something ideal These sen- 

suous shapes and sounds appear in art not merely for the sake of 
themselves . . . but with the aim, in this shape, of affording satis- 
faction to the higher spiritual interests, since they have the power 
to call forth from all the depths of consciousness a sound and 
echo in the spirit. (37-39) 

Contemporary policymakers in education do not have much truck with 
talk of “spirit.” But Hegel is careful to define spirit as self-consciousness seeking 
to find expression for all its potential. In other words, Hegel is first of all an 
educational theorist attempting to define the many distinctive ways we bring 
self-consciousness to what we construct as our places in the actual world. Basic 
modes of self-consciousness each require different kinds of assessment if they 
are not to collapse into something ultimately alien. Art asks that self-conscious- 
ness be directed to how the imagination can provide individual transformations 
of experience that take particular sensuous form but cannot be exhausted by 
straightforward description of that mode of appearance. As Richard Wollhcim 
demonstrated, art is not just a viewing of the sensuous but a seeing in to that 
sensuousness so that it cultivates self-consciousness of powers that we realize 
only through this mode of apprehension (45-49). Try imagining a range of 
experiences of joy without music or rapt attention without visual art. 
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Hegel’s third claim is the one most dependent on, and illustrative of, his 
idea of spirit because it depends on the possibility that self-consciousness can 
grow and intensify its focus by having itself as the object which it must try to 
understand. Most of us now are as suspicious of claims that anything can be 
an end in itself as we are claims about spirit — and for good reason since the 
two beliefs are very closely connected. But I think there are many experiences 
that we cannot sufficiently honor without a distinction between treating situa- 
tions as having ends beyond themselves and as having at least important 
dimensions in which the experience is an end in itself to be elaborated simply 
for the sake of what it affords self-consciousness. (This is easier if we admit 
that experiences can have different dimensions inviting different modes of 
apprehension and assessment.) 

The point is sufficiently important that I will indulge in some amateur 
philosophizing before returning to Shakespeare and the topic of assessment. 
We can treat any purposive action as having its end either outside itself or in 
itself. Attributing an end outside itself consigns the object of the action to the 
status of an instrument or tool that facilitates the accomplishment of some 
desire. But the object itself then ceases to interest — compare eating to fuel up 
with eating to relish some distinctive qualities in the food. (Or compare what 
one can call meat fishing with perfecting one’s fly-casting, even at the risk of 
going hungry.) Notice that when we choose to treat some process as an end in 
itself rather than as an instrument, we grant it the power to establish values, 
and we orient ourselves to acknowledging in practice the difference this 
makes. We can be happy with our fly-casting even if we catch no fish and 
remain hungry. We might contest someone’s assertions about ends in them- 
selves but we rarely feel free simply to override what they refer to for our own 
purposes. And we therefore give some concreteness to the notion of spirit as 
the capacity to dwell within conditions where we seek to intensify conscious- 
ness of who the self becomes as it renounces its habitual sense of treating the 
world in instrumental terms. 

Shakespeare probably wanted to intensify just this contrast between kinds 
of ends in the second speech of Cleopatra that I have chosen as an example. 
So please ask yourselves what is accomplished by Cleopatra saying “Not know 
me yet?” as the climax to her frustration and disappointment with Antony’s 
letting his shame at the military defeat turn into jealousy and self-pity. I think 
here she has to be fully self-conscious of the roles she has chosen to play 
because she may have only her self-consciousness to dignify what is likely to 
happen with Caesar’s victory. So she relics on the internal relationships that 
Shakespeare composes for his drama to find a self adequate to this moment. 
She experiences a close connection between trying to establish a self not gov- 
erned by prudential interests and the capacity of looking back on what she has 
been in the play to establish powers to maintain what can resist the prudential. 
Notice that her one line statement itself builds on a series of one line refusals 
in the scene that are all directed against Antony’s self-pitying theatrics. (Her 
richest moment may be her statement “Have you done yet” [3.13.155] that 
magisterially dismisses Antony’s letting himself indulge in the shame he feels 
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because of the contrast between what he “was” and what he now “is.”) 

Cleopatra’s “Not know me yet” also elaborates a much more comprehen- 
sive sense that the play is the thing wherein to establish a sense of character 
that one can bring to the actual world. What would it mean for Antony to 
know her at this moment? And by implication what would it mean for the 
audience to know her? Indeed what can we assess of that knowing — what out- 
come can we project and test? Probably our only answer can be that the 
knowledge has to reside in the quality of our reading and participating imagi- 
natively in what she composes as her character. Antony can not know her at 
this moment because he is so full of himself. He is doomed to be a bad reader. 
But he can show by contrast what good reading will probably involve. And 
that reading will not produce an object of knowledge. Cleopatra could not 
even produce that. Rather good reading will provide evidence that one has pro- 
visionally taken Cleopatra’s part so that one can give full imaginative credit to 
her ability to manipulate political situations and, more importantly, to be faithful 
in her fashion to the image both lovers have created of a kind of transcendent 
dignity of passion. 

The best readings of Antony and Cleopatra will not entirely submit to her 
capacity to control theatrical space. Rather they will know her also in a way 
that she resists knowing herself. They will know how tempted she is to find a 
way to reconcile with Caesar and how being a strumpet perhaps haunts her 
fears that success as an Egyptian queen may not qualify her to deal with those 
instrumentalist Romans who have built greater power. And they will know 
how desperate she is that Antony find his way back to the fragile but powerful 
myth of heroic lovers that binds them in what may be little more than illusion. 
In other words, these best readings devote themselves most fully to the various 
intense and intensive patterns of meaningfulness that the text establishes for 
and within its sensuous action. Assessing these readings then may well require 
an awareness of how students can ultimately develop a sense of texts as ends 
in themselves — not as escapes from the world but as the fullest possible means 
of honoring the intelligence the work brings to the world. 

Hegel makes a pretty fair spokesperson for the kind of reading that answers 
Cleopatra’s call for a distinctive kind of knowledge: 

Against this [the idea that “the work of art would have validity 
only as a useful tool”] we must maintain that art’s vocation is to 
unveil the truth in the form of sensuous artistic configuration to 
set forth the reconciled opposites . . . and so to have its end and 
aim in itself, in this very setting forth and unveiling. For other 
ends, like instruction, purification, bettering, financial gain, strug- 
gling for fame and honor, have nothing to do with the work of art 
as such, and do not determine its nature. (55) 

So long as we believe that it makes sense to confer a state of distinction on 
what cultures honor as works of art we will have to be leery of all claims 
about assessment that try to hold a variety of pedagogical practices to single 
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standards. And in the case of education for appreciation this problem is exac- 
erbated by the fact that any model of assessment responsive to the ideals 
informing the teaching will require an inordinate expense of money and time. 

First, assessing such teaching for aesthetic ends requires assessing the quality 
of classroom conversation. Then it entails finding objective means of evaluating 
the kind of student writing that tries to be articulate about what is involved in 
engaging texts as aesthetic experiences. I do not think that can be done under 
the same rubrics we use to assess student writing intended to convey information 
because in writing on aesthetic objects every turn in the argument involves a dis- 
play of sensibility that affects our sense of what the student can and cannot 
respond to. And the qualities of the thinking displayed cannot be presumed to 
be the same qualities we value for the economical and clear presentation of dis- 
cursive arguments. If students learn to be fully responsive to such plays they will 
want their response also to be at least in part something that they can feel 
reflects their own individual capacities to respond with affective intelligence to 
what moves them about their worlds. Even when papers on aesthetic objects 
take the form of clear arguments, the success of the argument depends to a 
large extent on an assessor’s distinctive grasp of the object addressed. The more 
the emphasis at every level on the importance of particularity, the less useful any 
general rubric for assessment — whether it be of the student or of the level of 
work being accomplished by the class. The abstractness of the instrument runs 
the risk of swallowing the distinctiveness of the performance. 

I hope I have demonstrated that there are good reasons professors con- 
centrating on the teaching of literature resist efforts to develop models of 
assessment that can be applied to their classes across the board. The risk 
involved is substantial because it encourages thinking that objective measures 
of any kind can be easily adapted for this fluid and often idiosyncratic process. 
It is difficult even to imagine any objective language of assessment that would 
not distort and dishonor the commitments to particularity that faculty want to 
encourage and develop — both in regard to texts and in regard to the students’ 
own senses of the qualities of discrimination they can bring to bear in their 
reading and in their writing. The effort to apply objective modes of assessment 
would influence how teachers being assessed taught their subject even though 
it does not emerge from study of that subject. So we would in effect be chang- 
ing the discipline in ways that few would explicitly defend, and we would be 
reducing the chances of students coming to experience what full cultural liter- 
acy can be. 

Most important, if we ever found an adequate way to assess the achieve- 
ment involved in writing about aesthetic experience, I cannot imagine that it 
would be very different in the long run from how any decent instructor evalu- 
ates writing by responding simply to criteria like attentiveness and imaginative- 
ness. And it would take examining just what these instructors examine, so it 
would involve an immense effort with little promise of significant rewards. It 
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would be far better to put what money we have into providing smaller classes 
so that instructors can assign more in-depth writing and develop more finely 
tuned assessments of how individuals are doing in the course. 


NOTES 

1 For powerful statements on possibilities of assessment see Wells, 

“Assessment without Angst” and Graff, “Assessment Changes Everything.” 

- I will use the concept of “appreciation” as short hand for “learning that 
focuses on aesthetic experience” even though technically appreciation is a 
specific mode of aesthetic experience where we dwell in the possibilities of 
judgment rather than make practical determinations. 

0 I take my sense of abstraction as form from Hegel’s notion of how philoso- 
phy brings out an internal necessity in relation to the phenomena under 
discussion. A successful concept is not content to provide descriptive termi- 
nology. Rather it develops an internal necessity so that the concept actually 
displays what we learn through it to be the necessary features of the con- 
cept. When we develop a concept elaborating relations among standard 
concerns about what makes art, we should be able to see that the claims so 
structure the field that they display their own fit and scope. 
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ASSESSMENT, LITERARY STUDY, AND DISCIPLINARY 
FUTURES 

LAURA J. ROSENTHAL 


MOST FACULTY MEMBERS IN LITERATURE departments had little 
familiarity with outcomes assessment until recently, and it seems to be coming 
at a bad time. It is no secret that the university is facing an unprecedented cri- 
sis, and that within the university the humanities are in a particular crisis, and 
that within the humanities the discipline of English literature has been in crises 
of various forms for many years (although perhaps unmatched by the current 
situation). A report from the Modern Language Association (MLA), for exam- 
ple, calculated a 27.5% drop in tenured or tenure-track jobs in English and a 
26.7% drop for foreign languages for 2009-10 (MLA, “Midyear Report”l); 
research recently conducted by an MLA task force showed that approximately 
66.6% of instructors in English and 57.1% in foreign languages are not 
tenured or on tenure-track lines (MLA and ADE 27). The percentage of stu- 
dents majoring in English has declined. Unlike in many other fields, the num- 
ber of doctorates earned has diminished (Lamont 59), and even given this 
drop, many critics (including many from within the discipline) think we are still 
overproducing. In addition to financial crises and apparent decreased appeal to 
students, literary study has also experienced several decades of internal conflict 
that has both energized and drained the field. Some of this conflict has attract- 
ed attention beyond the university walls, suggesting that the general public has 
a particular stake in what we do. Much of this publicity, however, has not been 
positive. As Louis Menand asks regarding the public reception of ideas pro- 
duced in literature departments, “How is it that humanists get painted into a 
cultural corner so that everything that a social or natural scientist says that is 
counterintuitive receives public genuflection, but literature professors are 
expected to do nothing but reaffirm common sense?” (10-11). And now, in 
addition to enduring public suspicion, fighting with crusty old colleagues or 
upstart young ones over curriculum, and competing for the students’ attention 
with the wireless internet that your university, which cannot afford to offer 
decent-sized classes, has nevertheless inflicted on your classroom, you need to 
produce reams of pointless information demonstrating that students are learn- 
ing something. Not only are you too busy to produce this pointless information, 
but there is something vaguely suspicious about the entire project. 
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For most faculty members in literary studies, outcomes assessment only 
became visible when accrediting agencies, pressured by the federal govern- 
ment during the Bush administration, began to require evidence of this prac- 
tice on campuses. This new development approximately followed No Child 
Left Behind (NCLB), sharing historical if not ideological ties to this much- 
criticized change in K-12 education. Few confronting outcomes assessment for 
the first time fail to connect the broad defunding of higher education and the 
particular crises faced by the humanities with the apparently increasing suspi- 
cion of those employed to teach in those fields. 

I will return to these crises and conflicts throughout the essay. But while it 
is certainly true that some governmental advocates of assessment (I am think- 
ing here in particular of the Spellings Commission report [United States 
Dept, of Education]) seem to have a fundamental distrust of higher education 
and a deaf ear for the subtle work of the humanities, and that the call to 
assess adds one more time-consuming project to increasingly demanding posi- 
tions with less and less support, I am going to make the counterintuitive sug- 
gestion in this essay that, whatever its origin, engagement with assessment can 
actually address the crisis within the discipline and beyond it in a productive 
way. Assessment, while coming to us as part of the problem, might be able to 
become part of the solution. In this essay, I will explore this possibility first by 
elaborating on “the problem” as both the ways in which literary study has suf- 
fered as a result of the general restructuring of the university and the internal 
conflicts that have made advocating for our discipline a particular challenge. I 
will then suggest how assessment projects can help us both understand and 
advocate for the value of literary studies. What I am addressing here is almost 
a “secondary” effect of assessment, its main purpose being the investigation of 
how well students are learning. Certainly the improvement of student learning 
remains the most important goal of any assessment project. The secondary 
effects, however, have perhaps attracted more attention in disciplinary venues 
in literary studies, and for this reason they deserve attention as well. While 
many critics have objected that outcomes assessment will necessarily miss the 
point of the discipline, I would like to suggest that it might help us articulate 
more clearly and more forcefully our “deliverables” — that is, what we offer 
students, the university, and the public. 

Before going forward, let me clarify what I mean by “assessment.” 

Roughly speaking, outcomes assessment as practiced on most campuses has 
two components: 1) fulfilling the demands of accountability; and 2) research 
into strategies for improving student learning. As Peter Ewell has pointed out, 
these separate components have separate goals and separate histories: long 
before accrediting agencies started asking for evidence of assessment, scholars 
had been engaged in a range of projects to figure out how well students were 
learning, from studies on an individual classroom to the development of learn- 
ing measurement instruments used across the country. I want to recognize 
how important this distinction continues to be. Nevertheless, while I am not 
specifically addressing issues of accountability in this essay, 1 I am using 
“assessment” throughout to address not just the practice of attempting to 
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improve student learning but the way those practices fit into institutions. 
Mostly in this essay, I am interested in the ways in which the practice of out- 
comes assessment might contribute productively to ongoing debates about the 
future of literary studies . - 

The Crises 

The larger crisis of the university has by now become familiar to many 
people both inside and outside of the academy. Because it provides the context 
for my argument about assessment and also for the reception of assessment by 
faculty members as part of new demands for accountability, it is worth briefly 
mentioning and distinguishing some of its different dimensions. Certainly 
higher education has been hit in profound ways by the recent recession, result- 
ing most dramatically in rising tuition costs, faculty and staff furloughs, cuts to 
important programs, hiring freezes, and larger class sizes. Considerable research 
suggests, however, that the current crisis has magnified a set of problems that 
have been building for a long time and that have resulted from a change in 
thinking about higher education as a public good that demands public sup- 
port, to thinking about higher education along the lines of a business, often 
referred to as the “corporate” model. In Shakespeare, Einstein, and the Bottom Line , 
David Kirp provides an excellent survey of the various ways in which business 
models based on profit and competition for prestige have reshaped higher 
education. Many institutions, for example, have attempted to repackage them- 
selves in ways that either de-emphasize academics or seek a kind of “niche” 
marketing to broaden their application pool and/ or attract more students. 
Further, departments have thrown themselves into “star treks with at best 
mixed results,” in which a series of high-profile hires are made in the hope of 
raising departmental rankings (Kirp 68). Many such attempts simply fail, Kirp 
points out, as these carefully constructed departments can quickly disintegrate 
when a few faculty stars accept other offers. But there is a cost even when they 
succeed. In New York University’s star-powered philosophy department, for 
example, the institution “sloughs off a great deal of its teaching onto part- 
timers,” as light teaching obligations are used to attract high-profile faculty 
(69). Such ventures, then, contribute to the crises in higher education by creat- 
ing a structure often sustained by underpaid adjuncts with limited job security. 

While Kirp looks at how universities have contributed to our current 
predicament by competing for rankings and prestige, Christopher Newfield 
offers a wide-ranging analysis of the erosion of the public commitment to 
higher education and its devastating effects on democracy. In Unmaking the 
Public University: The Forty-Year Assault on the Middle Class, Newfield explores not 
just the systematic defunding of public universities, but also the structural 
transformations within public universities on the model of private enterprise. 
Newfield’s larger point is to suggest how crucial public research universities 
have been to the expansion of the middle class, and how resistance to the mul- 
ticulturalism that public universities have tended to foster (by providing oppor- 
tunities for upward mobility to previously underrepresented kinds of students) 
has influenced their defunding. Without these institutions and broad access to 
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them, we lose a significant pathway to democratic participation across racial 
and ethnic groups. Further, Newfield shows that the corporate model favors the 
sciences for their potential to form partnerships with business, but that the 
humanities essentially subsidize these ventures by providing significant classroom 
education at a considerably lower cost. The “overhead” regularly charged to 
grants won by scientists rarely covers their price (195-207). Prestigious grants, 

then, while seeming to bring in money, in general need to be subsidized. Accor- 
ding to Newfield, institutions often turn to tuition income generated by the other 
side of campus in order to compensate for these expensive grants. The result of 
all of these policies has been a continued diminishing in the quality of public 
higher education. 

Two other studies worth mentioning in this context elaborate in differ- 
ent ways on the results of the situations that Kirp and Newfield describe 
by exploring the economics of instruction and faculty careers, particularly 
in the humanities. Frank Donoghue’s The Last Professors: The Corporate 
University and, the Fate of the Humanities and Marc Bousquet’s How the University 
Works: Higher Education and the Low- Wage Nation in different ways show how 
this same corporatization of the university has severely undermined facul- 
ty careers in the humanities, both linking this development to the broader 
casualization of labor since the Reagan era. Donoghue focuses specifically 
on the humanities, showing that the attack on humanistic learning has 
been going strong in the United States since “the emergence of America 
as an industrial power”: “Unregulated monopolistic capitalists such as 
[Andrew] Carnegie and [Richard Teller] Crane simply thought of the 
humanities as, literally, worthless” (xiii-xiv). For Donoghue, the danger to 
the humanities is posed less by the culture wars than by figures like Robert 
Zemsky, “whose whole body of work relentlessly poses the following ques- 
tion: Businesses and markets have always proven successful, so why don’t 
universities operate more like businesses?” (xv). Marc Bousquet explores 
one particular result of this business model: an explosion of reliance on 
adjunct faculty who teach courses without receiving benefits, a living wage, 
or job security. By placing high priority on producing student credit hours 
at the lowest possible cost, institutions of higher education in his view have 
become highly exploitative employers. While Donoghue sees no way out of 
this downward slide that will end with the elimination of tenure and thus 
the university as we know it, Bousquet wants to call attention to the cur- 
rent situation as a crisis in labor relations that can be addressed through 
strategic action. 

The restructuring of higher education documented by Kirp, Newfield, 
and Bousquet has affected every discipline. Perhaps, though, their arguments 
have a particular urgency for literary studies, which has a difficult time 
defending itself from an economic perspective and has also been the object of 
attack for its internal conflicts and reputed failures. (Surely it is no coincidence 
that Newfield, Bousquet, and Donoghue all find their disciplinary homes in 
English departments.) Literary studies as a discipline has for several years 
been facing challenges on two fronts: the economic woes that seem to place it 
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at a disadvantage, and concern over internal conflicts about the purpose of 
the discipline. So while some critics see literary studies as economically unpro- 
ductive or would reduce the field to instrumentality (English majors write 
good memos!), others have attacked the discipline for failing to defend its tra- 
ditional values. For many years conservative critics have been arguing that lit- 
erary studies has abandoned its mission by embracing theory at the expense of 
great books, and multiculturalism at the expense of aesthetic values. It is no 
surprise, then, to find these arguments converging. In a recent article in The 
America n Scholar , William M. Chace suggests that the decline of English depart- 
ments originates in its own disciplinary practices: 

At the root is the failure of departments of English across the 
country to champion, with passion, the books they teach and to 
make a strong case to undergraduates that the knowledge of 
those books and the tradition in which they exist is a human good 
in and of itself. What departments have done instead is dismem- 
ber the curriculum, drift away from the notion that historical 
chronology is important, and substitute for the books themselves 
a scattered array of secondary considerations (identity studies, 
abstruse theory, sexuality, film and popular culture). 

Chace acknowledges other factors, such as an increasing emphasis on “practi- 
cal” subjects and a great proportion of students attending public institutions, 
which have tended to offer less support to the liberal arts than private schools. 
Nevertheless, he sees the abandonment of literature as the primary source of 
the problem: “English has become less and less coherent as a discipline and, 
worse, has come near exhaustion as a scholarly pursuit. English departments 
have not responded energetically and resourcefully to the situation surround- 
ing them.” 

It would be easy to dismiss Chace’s essay as one more attack in a long line 
of them. After all, we have heard many of these objections before from 
Harold Bloom, Camille Paglia, and others. Donoghue argues that the culture 
wars are essentially over, and that worrying about whether your course 
includes Shakespeare or Aphra Behn amounts to rearranging the proverbial 
deck chairs on the Titanic. Certainly this issue cannot be settled here; I will 
only note before moving on that turning back is probably not an option any- 
way. Given all the research and exploration of writers and cultural expressions 
outside the “traditional” canon in the last decades, re-embracing one particu- 
lar group of authors would feel intellectually dishonest even if one were con- 
vinced by the potential strategic advantages of doing so. Restoration drama, 
to take one example, no longer makes any sense without Aphra Behn, not sim- 
ply because she is a woman but because she was the second most produced 
playwright in her time, greatly appreciated in her day, and a skilled craftsman 
with stunning insight into the complexity of her society and the human beings 
who negotiated it. Yet, just because returning to a mythical past of literary 
coherence would prove quixotic does not mean that we shouldn’t think about 
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whether it might be possible to improve the external standing of the discipline 
by looking inward. 

While Chace raises an old complaint, it holds new meaning in our chang- 
ing circumstances. Gerald Graff has argued persuasively that the most typical 
institutional response to departmental disagreements over theory and the 
canon has been to expand the curriculum by adding new courses, new fields, 
and new kinds of practitioners to departments ( Professing Literature ). This obser- 
vation is fully supported by both the MLA bibliography and (in my experience 
at least) institutional practice. For Graff this strategy, while accommodating 
different points of view, has generated its own problems: students encounter 
vastly different ideas and approaches in different classes within the same 
major, and most cannot sort through these contradictions on their own. In 
addition to these pedagogical challenges, we can also see, as critics often 
observe, that this model of accommodation depends on a proliferation of 
positions, scholarly journals, conferences, etc. Departments have not, in 
Graff’s view, tended to address the intellectual and pedagogical problems of 
this model; further, we seem to be facing a situation in which this kind of 
expansion poses particular challenges. 

While I do not, then, accept Chace ’s suggestions that an outdated cur- 
riculum would attract more students, it is nevertheless worth considering the 
possibility that critical self-reflection could be productive and that there is 
more to be done in the way of communicating the value of the discipline in 
ways that do not assume that this value is self-evident. In arguing this, I do 
not mean to imply that none of this is taking place already. The recent 
MLA/Teagle report represents one such effort, in which a team of scholars 
made a case for a particular set of disciplinary goals (“Report to the Teagle 
Foundation”). In fact, there been many attempts from many different angles 
to analyze our current challenges and point the discipline in particular direc- 
tions. What has attracted less attention, however, is what actually happens in 
the many classrooms in which literary studies take place. Chace complains, 
for example, that the Harvard undergraduate curriculum has replaced its sur- 
vey with a course of study based on “affinity groups” (“Arrivals,” “Poets,” 
“Diffusions,” and “Shakespeares”). For Chace, the weakness of this course of 
study is evident from the titles. Those titles alone, however, actually give us 
very little information. Do we know from them what or how well students are 
learning? Did we know what or how well they were learning in the surveys 
they replaced? Perhaps more to the point, what are they learning at the 
University of Massachusetts? At Massachusetts community colleges? 
Answering those important questions would require a different kind of 
research than a glimpse at the undergraduate catalogue. Both attacks and 
defenses of literature programs tend to be launched more often on the basis 
of what is (or should be) taught rather than what assessment helps us see as 
the more pressing issue of how students are learning and what kind of 
thinkers they can become. 

Outcomes assessment has been greeted with considerable suspicion in 
part, I believe, because it appears to be consistent with the corporate model, 
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thus alienating both traditionalists and non-traditionalists alike. It seems to be 
asking us to bloodlcssly quantify skills that a literature major will acquire and 
an employer might be looking for. It seems, at first glance, to ask us to reduce 
ineffability to instrumentality. A conscientious engagement with the questions 
that outcomes assessment can raise, however, holds the potential to move us in 
some productive directions. Specifically, thinking deeply and systematically 
about what we want students to learn from literary study, about how we hope 
they develop as they move through our programs, about what kinds of capaci- 
ties we would like them to develop, and about what kinds of inquiry they 
should become capable of pursuing will offer more clarity about the future of 
the discipline than these relatively unproductive debates over the canon. Shifting 
our focus in this way will also, I believe, offer the potential to help us advocate 
for our significance as a discipline and more clearly come to terms with the 
nature of our challenges. Assessment, however, has met with particular resist- 
ance in literary studies for reasons that make sense in the current context but 
that could ultimately prove counter-productive. Before elaborating on how 
assessment could help, then, I would like to explore the specific disciplinary 
panic that it tends to generate. 

Deliverables 

The crisis specific to literary study has many components, but at least part 
of it has to do with questions from within and suspicions from without regard- 
ing disciplinary purpose. To demonstrate this, it might be useful to turn to one 
of its less volatile aspects. Once I served on a multidisciplinary committee to 
distribute grant awards, and I remember feeling that other members were 
under-appreciating some very strong proposals from English. When ques- 
tioned about this, my colleagues objected to the obscurity of the English pro- 
posals, especially in regard to the crucial category of “impact.” Thus while I 
could understand the proposal from the evolutionary biologist on the genetic 
traits of albino cave worms — that is, I could tell you that the project involved 
albino cave worms and even, actually, why they mattered — the other commit- 
tee members had found it difficult to figure out even the topic of the English 
proposals and had no idea how funding them would produce, as they put it, 
“deliverables.” 

Michele Lamonf s recent book How Professors Think: Inside the Curious World 
of Academic Judgment, an engaging study of interdisciplinary grant proposal 
evaluation panels, suggests that this experience was not idiosyncratic: “English, 
the only field [of the five she studied] to show a decline in PhDs granted 
between 1995 and 2005, is also the discipline where the very concept of aca- 
demic excellence has come under the greatest attack” (59). Proposals from 
English have become less competitive in national grant competitions perhaps 
because, she speculates, of an internal relativism that has led to a “crisis of 
legitimation.” Similarly noting the declining status of English based on the 
diminishing number of undergraduate English majors, Louis Menand in 2005 
encouraged the colonization of other disciplinary objects as one positive step 
that literary scholars could take to enliven their work and expand their audi- 


189 



LITERARY STUDY, MEASUREMENT, AND THE SUBLIME: DISCIPLINARY ASSESSMENT 


ence (14). This advice seems to have been followed: Lamont suggests that 
English professors have attempted to address their “demographic decline” 
through cross-disciplinary efforts. This cross-disciplinarity, however, has in her 
view had the unintended effect of diminishing English’s competitiveness by 
producing work that scholars in other disciplines see as subpar attempts at his- 
tory or anthropology or sociology. Proposals from literature scholars, in other 
words, fail to express the unique qualities of the disciplinary approach to the 
material, which no longer remains defined by a particular set of texts. 

I believe that this disciplinary instability — what Lamont calls the “crisis of 
legitimation” — accounts in part for the way those in the field often find the 
call to assessment particularly vexing. If indeed the discipline has, as Graff 
suggests, expanded rather than sorted or confronted, then assessment poses a 
particular kind of challenge. The English department, of course, is not the 
only place on campus where one meets with distress over assessment; it seems, 
however, that assessment projects meet particular roadblocks in literary stud- 
ies. When he was MLA’s president, Graff cast his own advocacy of assessment 
as a significant departure from mainstream thought in the profession: “In the 
hundreds of faculty meetings in my forty-plus years of teaching, I never heard 
anyone ask how our department or college was doing at educating all its stu- 
dents” (“President’s Column” 3). Not only does Graff characterize assessment 
as a radical break from customary departmental practices in English depart- 
ments, but he devotes much of the column to countering a paper on assess- 
ment given by Michael Bennett, in which Bennett encouraged faculty to 
“resist” assessment because it is a “dodge from the real problems of the 
American educational system” (qtd. in Graff, “Assessment”). 

A version of Graff’s essay that appeared in Inside Higher Education generat- 
ed considerable debate, much of it attacking his support of assessment 
(although some praising it) (“Assessment Changes Everything”). Of particular 
interest in this context is a vigorous disagreement by Laurie Fendrich, who has 
elsewhere criticized assessment (“A Pedagogical Straitjacket”; “You Will Be 
Held Accountable”). Writing from outside the discipline of literary studies, 
Fendrich characterizes Graff’s interest in assessment as a form of reparation: 

Perhaps Professor Graff suffers from guilt over what’s happened 
to his own field — English — during the past several decades. After 
all, he’s witnessed the destruction of the study of the putative 
subject at hand in favor of theory. That’s a sad thing, and to the 
extent that he participated in it, he ought to feel guilty. For the 
rest of us who didn’t destroy our subjects, however, Outcomes 
Assessment is a wretched thing. (“Gerald Graff’s Essay”) 

Fendrich understands Graff’s embrace of assessment as a strategy for discipli- 
nary repair made necessary by his contributions to its destruction. 

From very different angles, however, both Fendrich ’s comments and Graff’s 
essays support the suggestion that assessment has met with, and perhaps by 
now has a reputation of meeting with, particular resistance in literary studies. 
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My own attempt to introduce assessment as a topic for discussion in the MLA’s 
Delegate Assembly, though ultimately successful, was initially met with some 
suspicion. Both at the meeting and before it, some colleagues expressed the 
concern that to hold an MLA-sanctioned discussion on this topic would legit- 
imize it, something the professional organization should be careful not to do. 

In other disciplines, by contrast, professional organizations themselves have for 
many years offered tools to help faculty with assessment (as actually, has the 
MLA, although perhaps less visibly than other organizations). The National 
Institute for Learning Outcomes Assessment (NILOA) has gathered together a 
considerable digital archive of advice and policy statements put out by various 
professional organizations representing many different disciplines.-^ The 
American Chemical Society’s Division of Chemical Education, for example, 
offers exams, laboratory assessment, and study materials. The Society’s web- 
page encourages instructors to return results from those exams so that the 
Society can compile statistical data. The National Communication Association 
offers its member a well-developed page of discipline-specific assessment 
resources, including criteria for assessment, guidelines for department assess- 
ment, assessment techniques and methods, and a conceptual framework for 
assessment. In addition to providing online resources for assessment, the 
American Political Science Association has recently published a book entitled 
Assessment in Political Science (Deardoff et al.). The American Historical 
Association does not post assessment material, but it lists criteria for standards 
in the field. The American Philosophical Association provides a careful expla- 
nation of assessment and examples of how to integrate outcome goals into 
courses. This document also addresses some of the concerns that members 
might have with assessment. The section for English lists two recourses: a link 
to position statements on digital literacy and assessment on the Conference on 
College Composition and Communication website and a link to a discussion of 
electronic portfolios sponsored by the National Council of Teachers of English. 
The editors of this web resource list nothing from the MLA. 

Surely there are other disciplinary societies that do not appear on the 
NILOA website and, as the essay by Feal, Laurence, and Olsen in this volume 
demonstrates, the Association of Departments of English (ADE) and the 
American Council on the Teaching of Foreign Languages (ACTFL) — both part 
of the MLA — have in fact been exploring assessment for many years. Neverthe- 
less, the NILOA website certainly gives the impression that other disciplinary 
organizations might be devoting more consistent attention to this issue. This gap 
may have part of its origins in skepticism. In a response to Graff’s position, Kim 
Emery has argued against engaging in assessment because the key to academic 
life is that “intellectual inquiry leads to unexpected places” and that assessment 
does the opposite, offering “control and containment in place of open-ended 
exploration” (259). While I think Emery is right to oppose standardization and 
that good teaching in the humanities must allow for the unexpected and that we 
need to preserve the possibilities of holding discussions in class that may not be 
easily digested by a general public, there is I think a silent qualifier in her goal of 
the “unexpected place,” for surely not any unexpected place shows student learn- 
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ing or helps move the class forward. She means, I think, a productive or at least a 
relevant unexpected place, yet declines to include such a qualifier. With the qualifi- 
er, however, we would be compelled to offer at least a working definition of what 
productive means and where the relevant places might be, which would in turn 
demand an acknowledgment of the invisible frame that shapes the debate itself. 

Like all work in the humanities, literary studies might be less systematic 
than math or physics, although in fairness many teachers of math and science 
might tell you that getting at student capacity to grasp the underlying meaning 
of equations and formulas poses the same kind of challenge. 4 The fluidity, 
instability, or even mystification of the discipline, however, has been noticed 
not just from traditionalists, but from more progressive angles as well. Graff 
describes it as undemocratic and even elitist ( Clueless in Academe). How can stu- 
dents without backgrounds rich in cultural capital ever hope to figure out how 
to succeed in a literature program unless we offer them clearly articulated 
goals? While not writing specifically about classroom learning, William B. 
Warner and Clifford Siskin have objected to what they see as the outright 
resistance to definition at the heart of the cultural studies project, which has 
been adopted by many literature departments precisely for, in their view, its 
lack of precision. They propose that cultural studies became so popular 
because it offered an umbrella under which a range of different interests could 
flourish but remain undefined. W T hile cultural studies claims an interdiscipli- 
nary reach, Warner and Siskin suggest that it nevertheless became palatable to 
English departments through its continued dependence on concerns tradition- 
al to literary studies. In their astute analysis, cultural studies combines an 
interest in “Culture” in the anthropological sense with a continuing commit- 
ment to “culture” in the sense of “high culture” (102). Cultural studies, they 
point out, has neither a well-defined methodology nor a clearly demarcated 
field of study, a point they take from Simon During’s own concession in his 
history of British cultural studies (qtd. in Warner and Siskin 102).^ By retain- 
ing “Culture” within the broader category of “culture,” English departments 
have not become interdisciplinary, but have instead expanded their discipli- 
nary objects. But while Menand, as mentioned earlier, argues that this expan- 
sion could invigorate the discipline, Warner and Siskin conclude that the 
appeal of cultural studies lies in its “Teflon” surfaces that made this approach 
so ambiguous that it is now time to abandon it altogether (104). 

While Fendrich and Chace see literary studies as a discipline that has 
destroyed its own object, then, Warner and Siskin see a constant process of 
reconstitution that takes place through the refusal to articulate central goals. In 
their view cultural studies needs to be stopped because it 1) has become for- 
mulaic and repetitive; 2) has not lived up to its promise to leave behind tradi- 
tional literary study; and 3) has neither a demarcated field nor a central 
methodology. It is (3) that is most relevant to this discussion: while not neces- 
sarily suggesting that English departments have destroyed their object of study, 
Warner and Siskin nevertheless identify and object to a lack of definition at 
the heart of the discipline as currently practiced. Ovcrreliance on the term 
“culture,” in their view, allows us to live comfortably with that lack of definition. 
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Assessment, however, does not. In spite of the usefulness or strategic 
avoidance of conflict that a capacious rubric like “cultural studies” may offer 
to the maintenance of working relations and scholarly practice, outcomes 
assessment ultimately demands both collaboration and serious thought about 
disciplinary purposes. 

Teaching as Disciplinary Practice 

The fact that assessment begins by defining certain goals constitutes its 
tremendous, even panic-inducing challenge; at the same time, this fact also 
holds its potential value to literary studies, as it offers the possibility of devel- 
oping significant insight into a discipline described by so many critics from 
both the left and the right as lacking a point. While assessment projects may 
begin in literature programs by identifying certain easily measurable skills 
(such as the ability to use proper citation), faculty will eventually need to ask 
searching questions about what they hope to pass along to their students and 
how they want to transform them. These goals will need to be explored col- 
laboratively. 

Graff, Warner, and Siskin do not argue, as do Chace and many others, 
that literary studies should return to the study of a narrow set of great works 
of literature (in the case of Warner and Siskin at least, quite the opposite). Yet 
they seem to be in agreement that intellectual progress and disciplinary health 
are not well served by an apparently prevailing resistance to definition. Begin- 
ning an assessment process does not necessarily bring disciplinary clarity, but 
in my experience it demands that certain invisible boundaries of a discipline 
be rendered visible, first to practitioners and then to students. If there really 
are no boundaries, then we’re probably not a discipline. (Parenthetically, I’ll 
say that there might be a case for this and that the universities of the future 
thus might not have English or history or art departments but just have 
humanities departments. We might consider whether or not this is desirable. 
Recently I have heard about many attempts to merge departments on various 
campuses, but no one seems very happy about them.) Like the prospect of 
being hanged in a fortnight in Samuel Johnson’s observation, assessment con- 
centrates the mind wonderfully. Most of our work, even work that we consider 
interdisciplinary, takes place within a set of invisible frameworks. Perhaps 
because the discipline of English has changed so rapidly or perhaps because it 
has been attacked so relentlessly, the prospect of bringing those frameworks to 
light seems particularly vexing. Yet this, I think, is exactly what we need to do. 
I am not implying, of course, that no one is thinking about those frameworks. 
Arguably the contrary: these issues about the direction of the discipline have 
attracted enormous attention in the last couple of decades. But while these 
discussions have brought considerable attention to what we teach, fewer have 
tackled the problems of how we teach and why we think students have some- 
thing to learn from our discipline. To get at these stickier questions, I think 
that reflections on these issues could be enhanced by a more broadly collabo- 
rative project in which disciplinary goals, values, and significant contributions 
emerge through attention to multiple assessment projects, in which a wide 
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range of teacher-scholars come to terms with and articulate the foundational 
goals of their courses and their programs, and which will lead to a more vig- 
orous and more transparent discussion of disciplinary aims for the future. This 
would involve better communication and more opportunities for pedagogical 
discussions at professional meetings, as well as serious consideration of how 
well we train graduate students for this important part of their jobs. It would 
also involve more attention to learning on individual campuses — not prizes for 
“great teachers,’' but collaboration in shared goals for student learning. We 
might discover a greater variety than most of us have considered, and/ or we 
might find more in common than we would have thought. Either way, there is 
probably more information out there about practice than ever. We need to 
supplement vigorous arguments from leading scholars about the direction of 
the discipline with a greater awareness of what is evolving in various locations 
“on the ground.” We might think about ways to “reverse-engineer” the disci- 
pline through multiple assessment projects as the process of revealing to our- 
selves and articulating to others what students passing through our classrooms 
learn. This is not the by-product of disciplinary knowledge; this is the discipline. 

It may be objected here that I have assumed that our disciplinary goals 
are the same as our learning outcomes goals, an assumption that would need 
to take into consideration the fact that we might not actually be teaching those 
colonized interdisciplinary objects that have entered our research. Certainly 
there are important ways that teaching and research will not be aligned: 
research projects often lead us to primary texts that only a specialist could 
love. Nevertheless, learning goals in our courses can help us see more clearly 
what we value and reveal to us what needs to be discovered or analyzed or 
challenged. They can expose gaps in knowledge, or the ways in which time- 
honored reading strategies need to be rethought, or the way a particular set of 
texts no longer addresses key issues the course is raising. Further, using assess- 
ment as inspiration for scholarship and disciplinary goals makes crystal clear 
the need for continuing research in the humanities. Students will not be 
engaged by outdated readings that do not speak to their experiences. The 
South Sea Bubble looks different now than it did before the recent financial 
crisis. Mr. Spectator’s claim to world citizenship has a different resonance for 
those raised in an age of globalization. 

So in short, I believe that assessment can help us reformulate and more 
clearly articulate our discipline from the ground up rather than the top down. 
There are many fine essays by insightful scholars suggesting where literary 
study needs to go, and scholarly trends can certainly be traced through book 
reviews, publishers’ lists, and the MLA bibliography. But what do we really 
know about our primary interface with the public and primary point of influ- 
ence? As Robert Scholes has observed, “Many scientists learn in order to pro- 
duce new objects and practical procedures. Most humanists learn in order to 
teach. It’s as simple as that” (8). As we engage interdisciplinary possibilities 
and work in universities structured by scientific careers, we can nevertheless 
only make a case for our continued existence if we take our difference into 
account as well. While the biologist may ultimately isolate a gene in albino 
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cave worms that helps a chemist produce a drug to manage skin cancer, it is 
the learner transformed, rendered capable of further self-directed inquiry, that 
will ultimately give us our “deliverable.” 

Reimagining an educated, critically conscious, aesthetically and ideologi- 
cally sensitive population as our “deliverable” will not only open us up to 
greater clarification of our disciplinary goals, but it will also put us in a better 
position to address the crises with which I began this essay. Most of these 
issues go considerably beyond the department level and involve decisions 
being made out of the hands of most literary scholars, but they are neverthe- 
less worth thinking about. Here are some preliminary thoughts about how 
assessment might contribute. First, assessment puts teaching more prominently 
on the radar screen, and a higher institutional priority of (or even attention to) 
student learning is generally good for the humanities. In the sciences, as 
Scholes convincingly observes, there is a greater separation between research 
and teaching. Universities, however, tend to be organized on the science model 
with no recognition of this crucial distinction. Second, in a world in which stu- 
dent learning and transformation become a priority, assessment projects can 
provide powerful arguments for thinking more carefully about employment 
conditions of all instructors. Most research suggests that student learning suf- 
fers when institutions fail to hire enough instructors with job security 
(Benjamin). This research needs to become more central to conversations 
about funding and institutional priorities. Third, if cultures of assessment 
developed in higher education, those “star treks” discussed by Kirp might not 
have quite the same kind of priority. 

Certainly, there are ways that assessment can be used as micromanage- 
ment and to develop destructive forms of standardization. Collaboration 
within programs can improve student learning, but instructors still need to 
retain creative control of their classes, to allow for unexpected possibilities, 
and to appreciate student accomplishments that they may not have antici- 
pated in the rubric. 6 Sometimes assessment seems like pointless bureaucracy 
or worse, like a higher education version of No Child Left Behind. But if 
No Child Left Behind taught us anything, it was that “non-response,” as 
David Mazella argues in this volume, “is the worst possible choice” (237). 
Fuller engagement with the demands of assessment may help us avoid its 
more troubling potential effects. 7 Departments can always find ways to look 
like they are assessing to satisfy the pressures of accountability without hav- 
ing the kinds of conversations that a good assessment process can inspire. 

But just like the student who spends more intellectual and creative energy 
figuring out how to get around your syllabus than he would need to spend 
just doing the work, departments probably end up creating more bother for 
themselves when they fake assessment rather than genuinely engage transfor- 
mative learning. Good assessment practices could help us productively con- 
front the core of the discipline, its particular challenges, and what it might 
still has to offer. 
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NOTES 

1 See Graff and Birkenstein in this volume for such a discussion. 

O 

4 English and foreign language departments, of course, teach much more 
than literary studies: they teach rhetoric, language skill, composition, writ- 
ing across the disciplines and creative writing, to name a few. For present 
purposes, however, I am focusing on literary studies, a phrase I use as 
shorthand to cover both the traditional study of literature and various 
forms of cultural studies now taught in English departments. Whatever 
the conflicts between “literary” and “cultural” studies, both tend to see 
themselves as part of the same discipline. 

0 The list of disciplines with their professional resources is found on the 
NILOA website. 

^ Many examples of this can be found in Ken Bain’s What the Best College 
Teachers Do. 

^ Warner and Siskin 102, citing Simon During, cd., The Cultural Studies 
Reader (London: Routledge, 1993). 

® On this issue, see Sarah Goodwin’s essay in this volume. 

^ See the essay in this volume by Pat Belanoff and Tina Good for a power- 
ful elaboration of this proposition. 
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THE FUTURE OF LITERARY CRITICISM: 
ASSESSMENT, THE CURRICULARIZED CLASSROOM, 
AND THICK READING 

CHARLES M. TUNG 


MOST REVIEWS OF THE STATE of literary studies are some mixture of 
elegy and coroner’s report. As Marjorie Perloff notes in “Crisis in the Humanities? 
Reconfiguring Literary Study for the Twenty-First Century” “one of our most 
common genres today is the epitaph for the humanities” (668). But in recent 
years the writing on the wall — or headstone — has become more emphatic. In 
light of Perloff’s arguments as the 2006 Modern Language Association (MLA) 
President, one might say that the word history , which at first marked the temporal 
situation of all knowledge once kept out of time’s way, now spells the end, not of 
a particular way of practicing the discipline, but of practicing the discipline at all. 
As her “Presidential Address 2006” warned, literature departments must confront 
the current predicament in which “the literary, if it matters at all, is always sec- 
ondary; it has at best an instrumental value” (655). 

What began as a methodological or theoretical difference seems now to 
have become an institutional and professional problem. Historicism, the very 
thing that was to make literature important, useful, and worldly, has snuffed out 
literature’s bid for social relevance by revealing to us how inadequate literature 
is as a definitive, non-trivial expression of the historical or as an effective inter- 
vention in it. Moreover, the decentering of ahistorical approaches by an 
increasingly distant reading appears to have undermined our institutional and 
disciplinary footing. As Jane Gallop claims in “The Historicization of Literary 
Studies and the Fate of Close Reading,” “we seem to have given up precisely 
what the historians envied and to have settled into a permanent position of 
inferiority. To me this looks like disciplinary suicide” (184). 

According to Perloff’s “Presidential Address,” if we survive this felo-de-se, 
what remains of us will be compelled by administrators to lead lives of 
comp/ rhet desperation. Literary studies, undone by its own zealous interdisci- 
plinarity and meretricious self-obliteration, is now facing a future in which 
English departments are asked to “concentrate on the study of composition 
and rhetoric,” the practical part of our discipline that has less trouble convinc- 
ing the world that we “really do teach students things they need to know” 

(656). 1 This revaluation of the usefulness of literary studies within the univer- 
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sity has already been expressed by recent rounds of the job market in which, 
as William Deresiewicz complained, “The lion’s share of positions [lias been] 
in rhetoric and composition. That is, not in a field of literature at all but in 
the teaching of expository writing, the ‘service’ component of an English 
department’s role within the university.”-^ 

But perhaps it is here — in the realm of curriculum, classroom, and critical 
reading practices — that we can begin to rethink the “common theoretical 
rationale” that Gerald Graff has found lacking in our discipline (“Future” 

258), and re-establish our footing and justification in the twenty-first-century 
university. Perloff’s view suggests that composition and rhetoric might be the 
last tree behind which the truly literary is hiding from “the increasingly socio- 
logical, political, and anthropological emphasis of English studies” (“President’s 
Column” 4). 3 But I think that insisting on the relation between low “service” 
and the loftier goals of literary studies might help us frame our work different- 
ly. It might help us theorize what we do more broadly and more pragmatically, 
and allow us to recast the speculative problem of the future of literary criti- 
cism as the institutional problem of the future of the literary curriculum. After 
all, “the actual state of the profession,” as James Sullivan pointed out in a 
brief response to Perloff’s address, shows us that “overwhelmingly, English 
departments are composition departments” also (Sullivan 255). 4 

In this essay, I would like to explore the nature of this also and its impact 
on the prospects for literary studies. Obviously, this exploration will not yield 
new methods for twenty-first-century knowledge production. But it will involve 
several unexpected moves that might shape how we think about the future of 
literary criticism’s interpretive project and the reasons why it is worth practic- 
ing. First, I will suggest that literary studies learn from composition’s openness 
to assessment so that we can begin the communal work of clarifying learning 
outcomes for literary studies in our particular departments. Guided by these 
outcomes, which are inevitably grounded in the conflicting practices and val- 
ues of our profession, we can then use the assessment principle of “backward 
design” to rebuild our curricula. Not only would the reverse-engineering of the 
literature major help undo our exaggerated sense of the difference between the 
autarkic fields of literary studies and the purportedly remedial mechanics of 
writing courses, but it might change our pedagogical aims. This change consti- 
tutes my second point. If we respond to Graff’s call as the 2008 MLA 
President to remedy the isolationism of specialization and the individual 
course, then our common work, which produces what I will call the “curricu- 
larized classroom,” might recast our basic activity — writing readings — as nei- 
ther close, nor distant, but thick. In other words, as composition and research 
come closer together, our pedagogical aims might shift from the exposure of 
students to disparate swaths of subject matter, or the recruitment of them into 
particular territories or “schools,” to the development of readers who can 
compose readings out of the fullest range of methods and the stakes they can 
articulate in different ways of seeing. Third, since this turn toward the curricu- 
larized classroom would entail a shift of theoretical focus from both the notion 
of the text and historicist decenterings of it, to the capacious form of writing 
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and response that we teach, as well as the ideals informing those argumenta- 
tive negotiations of disagreement and difference, I will hold out the possibility 
that we can reconsolidate the discipline by theorizing literary studies in terms 
of the widest range of critical practices that our curricula can bear, and that 
we can reassert the discipline’s value in a way that does not confuse chastening 
self-reflection with the self-abnegation that Pcrloff fears. As I hope to show, my 
suggestion for literary studies does not aspire to a partisan position in the rou- 
tine theoretical battles in our field, but rather to a multipartisan and inclusive 
recasting of our already plural discipline. Against Jonathan Culler’s long- 
standing call for the end of interpretation as the task of literary criticism, reit- 
erated most recently in his book, The Literary in Theory, I will propose an 
expanded hermeneutic project — thick reading — that might justify, secure, and 
reproduce literary studies in as much of its pcrspectival multiplicity as possi- 
ble, without sacrificing cither Culler’s concern for the historical and ideological 
“mechanisms that produce meaning in social and cultural life” (248), or the 
modes of inquiring, understanding, and experiencing that we idealize as readers. 

I 

Assessment has always partnered better with composition than with litera- 
ture. But what would happen if we joined composition in using assessment 
judiciously to help us refocus concern on the survival, viability, and strength of 
literary studies? As I move toward a discussion of core disciplinary practices, I 
want to hazard what I know will be an unpopular proposal: assessment can 
help us secure literature’s distinctive place in the university by getting us to 
examine what we are doing in our particular communities, by asking us to rec- 
oncile what we do with the ideals of the literary education we envision, and by 
affording us the space in which to align the goals and critical practices of liter- 
ary studies with justifications for their continued employment. By requiring us 
to consider the relations between our individual specializations and our collec- 
tive and agonistic learning outcomes, assessment can help us construct a future 
in which our research agendas are not so divorced from our pedagogies, which 
confirm as they count on our idealizations about the value of our reading and 
writing. If this were to happen, it would be possible to include the classroom 
as a compelling site of disciplinary formation. 

For most of us in literary studies, assessment seems the least likely to help 
us flourish. We sense in it dangers ranging from the disciplinary control of 
knowledge work to the nineteenth-century task of inducting novices into cul- 
ture in order to reproduce a heritage. Consider the controversy created by 
Graff’s presidential column in the Spring 2008 ALLA Newsletter (which he later 
published on the MLA blog “From the President” and on the Inside Higher Ed 
website). In this piece, “Assessment Changes Everything,” Graff proposed, as a 
solution to the many institutional ills diagnosed in his books Professing Literature 
and Clueless in Academe, a shift away from the “Best-Student Fetish” and “Great- 
Teacher Fetish” to an educational model in which we collectively articulate the 
ways we want to help all students learn, and then pay attention to what and 
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how wc do (3). Such a shift must take place at the departmental and profes- 
sional level because, in spite of the presence of great individual teachers, stu- 
dent learning as a whole suffers in atomized curricula of disconnected courses. 
Assessment, argues Graff, obliges us “to correlate and align our courses,” to 
“operate not as classroom divas and prima donnas but as team players” (4), in 
order to prevent the “mixed-message curriculum” and the inhibition of argu- 
mentative engagement that results from the university’s isolation of intellectual 
diversity (3). 

From colleagues on both the left and right, Graff’s argument immediately 
drew fierce opposition for the way the assessment of our practices projects 
clear ideological messages. On the MLA blog, Russell Potter expressed disap- 
pointment at hearing “the President of MLA mouth such inanities as ‘learning 
outcomes assessment,”’ since “education cannot be measured, quantified, or 

defined to everyone’s satisfaction [Wjhen all is said and done, what we do 

is not something that we can ‘know.’” 5 Later, Barbara Foley called assessment 
conformist and conservative, because it is driven by “administrators — and, 
behind them, boards of governors and trustees — to make higher education 

more productive for U.S. capital in the era of intensified global competition 

OA [outcomes assessment] is basically No Child Left Behind as applied to 
higher education.” Michael Bennett logged on to confirm these contexts in 
which assessment not only appears as “an organized effort to discipline faculty, 
staff, and students,” but also functions as a distraction from the larger issue of 
“the casualization and corporatization of academic labor.” And on the Inside 
Higher Ed website, Laurie Fendrich likened Graff’s vision of assessment to 
“Maoist indoctrination,” an effect that her Chronicle of Higher Education essay, “A 
Pedagogical Straitjacket,” described as a consequence of the humanities’ self- 
inflicted demise: “we in higher education . . . have, of course, brought this 
plague of pedagogical bean counters upon ourselves. We’ve spent the past 
half-century merrily ‘deconstructing’ our subjects and declaring that the idea 
of a knowablc core of what we teach is null and void.” 

Assessment as murdering to dissect, as misplaced empiricist reification, as 
politically conservative external imposition, as preparation for the beehive of 
global capital, as disciplining mechanism and strategic distraction, as Maoist 
indoctrination, as both guilt for and crime against the humanities — assessment 
seems to be the worst possible candidate for the rethinking of our discipline. 
Because there are indeed odious forms of assessment — reductive, external 
measurements that differ from the proprioception of particular discourse com- 
munities — it is not hard to understand the reaction to the poking and probing 
of literature, which can cause a colleague’s face to darken and fall like the holy 
countenance in Caravaggio’s The Incredulity of St. Thomas. From an equally 
scandalized position, Thomas becomes a centurion in the service of power, 
money, and official knowledges. 

But these reactions are based on the uses and understandings of both 
assessment and literary studies that the profession has the potential to modify 
to its advantage. While empiricist reckonings might be dcpressingly frequent 
and the conservative contexts ubiquitous, many departments have followed 
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successfully Sherry Linkon’s recommendation to “change the way assessment 
is practiced and understood” by making it serve “the holistic nature of a good 
college education,” as well as using it to solidify the discipline’s position in a 
competitive and underfunded environment. For instance, my own department 
has resisted reductive psychometric models in favor of what my colleagues 
John Bean, Theresa Earenfight, and David Carrithers have called “discourse- 
based assessment” (7). This style of practicing assessment draws on Barbara 
Walvoord’s “emphasis on the course-embedded assignment and on the profes- 
sional expertise of the individual professor, whose experience in grading stu- 
dent work is the foundational assessment act” (Bean et al. 6-7). Remaining 
grounded in their own experience as teachers, individual department mem- 
bers come together to discuss and make explicit their projections of discipli- 
nary competence informing their grading of student performances. The col- 
lective articulation of the interpretive and argumentative moves they expect in 
the work of seniors leads to the formulation of general departmental projec- 
tions: we ask what the end goals of our curriculum are, what we want our stu- 
dents to be able to do, and how that ability can be made visible and evaluated 
at different stages in the variety of curricular arcs we offer. 

Obviously this approach to assessment initiates a long and difficult 
process, the specter of which is often enough to make the agonistic articula- 
tion of curricular outcomes seem like an antagonistic impossibility But these 
departmental discussions can also be surprisingly fruitful and engaging in their 
focus on how we want our seniors to “do” the major — the kind of capstone 
seminar paper or undergraduate research conference paper, for example, that 
we might want to see them write. Once a set of broad outcomes is formulated 
by a department based on its range of in-house practices and disciplinary 
ideals, then faculty members can begin to address the equally challenging 
issue: how do they work backward from these outcomes to establish the paths 
leading up to these goals? What are the structures that need to be created — 
assignments, sequences of assignments, flexible relations among courses — in 
order to help literature majors build the strength to reach these objectives? 
Faculty must work together to analyze specific student exercises within specific 
courses at specific points in the curriculum. These analyses of the curricular 
location of particular skills and assignments help the department to use “back- 
ward design” to coordinate assignments and modify the relations among 
courses. The construction of visible paths to the ends of the major helps stu- 
dents learn to become the readers and writers who can engage the complicat- 
ed objects in our particular fields of expertise. 

I do not have the space here to provide examples of the assignment 
sequences and relations among courses produced by my own department’s 
process. In any case, the sequence and relations will always differ according to 
departments’ specific and diverse constellations of faculty. 6 Indeed, the partic- 
ular sequence of skills and methods is less important than the collective effort 
to discuss and make explicit to students a rationale for their learning different 
ways of complicating texts from different teachers (as opposed to their check- 
ing off coverage requirements). Moreover, in light of the ideal of thick reading 
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that emerges out of communal efforts, the order of our courses is designed not 
so much to produce a progression from basic to advanced, in which develop- 
ment is construed as “simply getting better at the same task over a period of 
time” (Carroll 23). It is rather to create situations in which students learn to 
see increasingly the thickness and complexity of signifying objects and “to 
write in forms more diverse and complex than those they could produce when 
they entered college” (23). My own community’s discussions produced an 
agreement that we did not want students simply to get better and better at, 
say, Marxist demystifications or structuralist analyses, but to learn to construct 
thick, researched, argumentative readings of signifying objects that could enter 
into, for instance, the perspectives of a Marxist or structuralist, while simulta- 
neously pushing against the views that might be held by our department’s 
postcolonialist or new media specialist. Or vice versa — the configurations are 
not prescribable. 

The payoff for going to the trouble of curricularizing the classroom and 
formulating outcomes that we track and substantiate over the course of our 
programs is clearly not just pedagogical but disciplinary. To adopt the simple 
logic of discourse-based assessment — that the collective projection of any goal 
is made realizable and improvable by the articulation of a process to meet it, 
followed by self-reflection on what might be modified for future iterations — is 
also to work together to situate the kinds of reading, thinking, and writing we 
teach in relation to our projections of the discipline as a whole and its survival 
instead of just the content we cover. Refusing to let curricular outcomes con- 
tinue to go without saying would mean articulating what we are doing to 
achieve the kind of majors who will benefit from and carry on our distinctive 
type of inquiry. And saying what goes without saying involves us in the negoti- 
ation of disagreement about the relationship between what we are doing, what 
we idealize in that doing, and what those idealizations might blind us to. This 
negotiation, which is exemplified in the many different shapes of thick read- 
ing, is fundamental to inquiry in the humanities, and, in combination with the kind 
of seeing upon which it relies, may be one of the best ideals of literary studies. 

I will say more on this reading and seeing below, but for now I want to 
note how hard it is to calculate the advantage of this ideal in the present envi- 
ronment. Even if we are able to change the way we think about assessment, 
the responses to Graff imply that we see the discipline as either something 
that cannot be known, or a body of knowledge whose marginalized position 
makes it susceptible to, or helps it evade and subvert, the regulatory grasp of 
master discourses. As a departmentalized field, literary studies has always 
seemed to be fighting off an cpistemic culture in which our knowledge of 
things ultimately becomes a measure of their value and a disclosure of fields 
of power. According to Daniel Green, this situation explains why the New 
Critics gave “the guardianship of literature to the academy in the first place,” 
a move “designed to establish literature and literary criticism as indispensable 
elements in the academic curriculum” (72; 74). In his view, the cpistemic 
establishment of literature in the academy backfired, resulting ultimately in lit- 
erature’s trivialization and commodification in “the twenty-first-century corpo- 
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ratized university with its emphasis on ‘product’” (72). The New Criticism’s 
“most lasting effect,” says Green, “may have been to demonstrate they [litera- 
ture and literary criticism] are entirely inessential to academic study” (74). 

But perhaps literary studies now is faced with the necessity, or opportunity, 
of tactically rethinking its content-focused goals of producing unruly, “useless” 
knowledges or highly relevant socio-historical facts. It certainly ought to con- 
tinue resisting the urge, created by the New Critical survivalism described by 
Green, to justify the discipline in terms of mystical non-discursive truths. 

None of these has worked in securing literature’s place in the university and 
general culture. Moreover, a strategic shift in disciplinary rationale does not 
necessarily entail any wholesale rejection of either the epistemological enter- 
prise of literary criticism or the value of aesthetic experience. Criticism can 
continue to generate knowledge of what texts say and what signifying mecha- 
nisms they employ. And even if one believes such knowledge is inadequate to 
the powers and intensities not containable “in” the text itself, it is nevertheless 
still the case that being “against interpretation” forecloses the possibility of 
getting to the place where we might recognize the value of something not 
entirely mappablc by the instruments we have. The student who wants only to 
sit in quiet reverence before the unknowability of art needs a teacher, not so 
much to be disabused of the reverence, but to be given a process leading up to 
it. Of course it is possible to have an experience and miss the meaning, but 
ideally one returns continually to a puzzle in order to make meaning a part of 
the significance, and significance a part of the experience. 

II 

As I mentioned in the previous section, literature professors have provided 
many reasons to fear and resist assessment. The professional payoff for using 
assessment might make us apologists for the knowledge factory and blind us to 
the reasons why historicization and the bid for social relevance have been dis- 
solving literary studies into both social science and composition. The class- 
room and curriculum might form a discipline whose shape is drawn entirely 
by power and repressions. From another perspective, the classroom and the 
institution of the modern university have always been poor homes for litera- 
ture, and strengthening its place there would only ensure its reduction to the 
knowledge regulated by departmentalization. But I think that assessment’s 
emphasis on the curricularizcd classroom can help us secure our place in a 
quite inclusive way — without sacrificing our distinctive justification, which is 
based not only on knowledge, but on the humanistic understanding and aes- 
thetic experience produced by literary reading. It can remedy some of the lan- 
guor of our self-consuming situation without asking us to surrender our ability 
to criticize overly narrow definitions of the discipline. 

Discourse-based assessment does this by challenging faculty to construct 
complex and capacious idealizations that are provisional and revisable rather 
than timeless and universal. These idealizations are not only the learning out- 
comes and curricular goals of our pedagogy, but also that which arises out of 
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them — the continual projections of and arguments about what deserves to be 
retaught, improved, and recast under shifting conditions. In my argument, 
what is worthy of continued life is not just a content or a skill, but the con- 
struction of and openness to perspectival multiplicity, and the agonistic 
engagement of difference through argument common to interpretive fields in 
the humanities. Well-argued, multifaceted disagreement is one of our best 
goals, since it requires an aspectual seeing that is central to literary studies. 

Without idealizations, and perhaps without the particular idealization 
above, the profession’s ability to maintain or re-establish its footing in the uni- 
versity becomes unlikely. We would remain in the condition that Elizabeth 
Renker, in her recent book, The Origins of American Literature Studies: An 
Institutional History , calls “post-curricular” (127). An effect of the wikipedifica- 
tion of knowledge and the increasing power of the student as consumer, post- 
curricularity is nearly finished displacing the old “top-down, scholar-driven 
professional model” of the discipline (127). In the twenty-first century, think- 
ing has become wiki-wiki work, libraries can no longer afford to buy books, 
and universities find much of the humanities lacking in even the rhetoric of 
value that might justify continued investment. 

To resist the transformation of the discipline into a factory outlet or show- 
room, to value the democratization of knowledge without confusing it with atom- 
ism, we could begin exploring the classroom, not the “top-down” one in which a 
field is “covered,” but the site informed by the idealizations and justifications of 
our various interpretive practices as they relate to and complicate one another. 
Indeed, if one accepts Graff’s descriptions of departmental life, let alone intellec- 
tual culture, it may be that a refusal to have these discussions that assess our col- 
lective institutional purpose and presence (as a set of intricately articulated and 
conflictual processes) is even more of a capitulation to the circumstances in which 
our “patterned isolation” is exploited by the corporatized university. 7 

The curricularized classroom, which stands as a figure for the skills and 
ideals we teach, ought to be worth risking the ideological dangers we fear. Its 
position in a process that aims to repeat itself, not as a static reproduction but 
as an agonistic, self-revising, and ever more capacious departmental and disci- 
plinary conversation, constitutes the reason why “teaching is central to the 
humanities,” as Robert Scholes pointed out in Profession 2005’ s “Presidential 
Forum: The Future of the Humanities”: “It is one place where our scholarship 
manifests itself — or should. Many scientists learn in order to produce new 
objects and practical procedures. Most humanists learn in order to teach. It’s 
as simple as that” (8): 8 In other words, learning in the humanities gets its value 
by virtue of the collective idealizations that make it worth sharing, rather than 
by the production of novelty or practical use per se. Although we usually think 
of teaching as introducing and establishing a field, and research as enlarging 
and complicating that field by the production of new knowledge, the humanis- 
tic interconnection of the two transforms the classroom into the space in 
which our practices and new knowledges are in a mutually modifying dialogue 
with those explicit curricular arguments about what is worth sharing and 
reproducing in and as a discipline. 9 
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III 

The future of literary criticism promised by making curricularizcd teach- 
ing more central to our professional self-understanding seems to go against all 
the major theoretical proposals for the discipline today Many follow Fredric 
Jameson’s suggestion that “literary criticism is or should be a theoretical kind 
of symptomatology” (407). Franco Moretti asks us to make “a little pact with 
the devil: we know how to read texts, now let’s learn how not to read them” 
(57). Perloff, as MLA President, calls for a return to poetics, as does Culler in 
The Literary in Theory. By contrast, the argumentative reading that results from 
communal assessments and curricular projections maintains and expands the 
fundamentally hermeneutic nature of literary studies. While my suggestion 
seems like a partisan one in light of the numerous critical and theoretical bat- 
tles waged in our field, it is in fact a pluralist, multipartisan position. Thick 
reading promises no earth-shatteringly new method or knowledge, but it offers 
distinct advantages, since it maintains literary studies’ focus on significant 
objects and structures without giving up literature’s extra-textual and interdis- 
ciplinary concerns. In the remainder of this essay, I would like to describe the 
advantages that a thick reading built on the curricularizcd classroom offers 
over distant reading, which is most often focused on history, and over poetics, 
which emphasizes mechanisms of signification. 

Although there can be no one model for thick reading, I want to rough it 
in as a compositional-interpretive practice and suggest its benefits by providing 
an oversimplified sketch of what I think is a very common lower-division path 
for students. At the start of a first-year class, a student will often see literary 
criticism’s best work as little more than personal impression or simple plot 
summary. Since the student has no sense of how the literary or cultural object 
functions as complex set of devices, the teacher tells the student to quit asking 
what the text is saying and start asking how the text is saying it. Perhaps the 
next teacher emphasizes prosody; the next, a theme played out in figures of 
speech and thought. The student keeps hearing the phrase “close reading.” At 
this level, the form of the student’s paper starts to shift from “The poem is 
about X, which I think is depressing” and “The author means X,” to “The 
poem is using Y and doing Z” or “The poem is using Y and doing Z to say 
X.” Soon thereafter, the student takes some survey lectures and several inter- 
esting courses organized around a historical moment or cultural node. The 
teachers invite the student to consider not only the devices but also the materi- 
al and cultural contexts in which the text’s saying, and the reader’s personal 
impressions, acquire a particular inflection at a specific time. For the majority 
of students (and not the high-achieving few who flourish because they are already 
able to detect invisible curricular connections), papers often begin to look like 
this: “This novel draws on a bunch of other writers and the Bible,” or “This poem 
is all about the French Revolution,” or the supremely indexical “In the text there 
is the theme of the machine uprising and the erasure of the human.” 

Depending on what might be taking place or what happens next, the 
direction in which the student is going is not so bad. The close reading to 
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which many students are still introduced seems, as Heather Murray notes, 
“basic to the pedagogic practice we value most highly, the detailed discussion’' 
(195). And Gallop, who has asked us to return to close reading from histori- 
cism’s “more authoritarian model of transmitting preprocessed knowledge,” 
says that it makes active learning possible (184). But it is a part of the history 
of our discipline that the special knowledge derived from the study of how a 
text works could not establish why the activity mattered, except to say that it 
somehow created a replacement for religion’s truths. Since this contraction 
drew criticism away from the noise of the world, contextualizations of all 
kinds came to rescue literary studies from the vacuum that was simultaneously 
irrelevant and politically reactionary. So where does the student go now? 

It is easy to see why symptomatology and distant reading drew an oppor- 
tunistic lesson about relevance from modernism — a lesson whose wisdom has 
turned out to be, in Lawrence Rainey’s words, that “the best reading of a work 
may, on some occasions, be one that does not read it at all” ( Institutions 106). 

But this wisdom is too costly. While close reading, as Charles Alticri puts it, 

“had come to prefer text to act (or Brooks to Burke) so that it could not ade- 
quately open itself to the range of human interests [generating efforts at liter- 
ary expression and understanding]” (“Taking Lyrics Literally” 259), the new 
distant reading’s zoom out from the well-wrought urn similarly loses its justifi- 
cation for why cultural phenomena like literature ought to command attention 
over anything else that opens onto underlying contextual conditions. 1 ® The 
road that began with New Critical explications of the text itself and ends with 
Jameson’s “mode-of-production analysis” (408) leads us to a justification of 
method by way of important historical stakes and gives us some explanatory 
purchase on our objects. But it also seems to make literary studies largely beside 
the point. Hyperopic not-rcading no longer needs literature as a crucial eviden- 
tiary site, nor, more importantly, the criticism required to engage it. And when 
it does, as Moretti provocatively shows us in “Conjectures on World Literature,” 
the analysis, contra Cleanth Brooks, is almost entirely footnotes. 

Both Perloff and Culler have rejected symptomatology, and both call, 
from different directions, for a return to poetics as the answer to the problem 
of our self-consuming institutional space. 11 For Perloff, poetics’ focus on the 
materiality of linguistic constructions — on “how language actually works and 
what it does” (“Crisis” 671) — can restore literary studies’ disciplinary identity 
from its epistemically oriented, overly thematizing identifications with history, 
sociology, and anthropology. For Culler, literary studies ought to give up “the 
lure of interpretation” for the structuralist investigation of signifying systems 
and mechanisms ( Literary 230), which would maintain literary studies’ pivotal 
position in the interdisciplinary linguistic turn. Together, both of these strate- 
gies have the advantage of emphasizing what literature does, and what textual 
and cultural work it performs, as opposed to what information it bears. They 
thereby resist the seductions of the interpretive project and the repetitious the- 
matic criticism that comes from our desire to extract messages from texts. 

This interpretive project captured literary studies in the first place because 
“if people want to study works of literature, it is generally because they think 
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these works have important things to tell them and they want to know what 
those things are” (Culler, Literary 230). But the will to knowledge of the mes- 
sage too often committed readers to the narrow hermeneutics of intended 
meanings, blinding them to the way meanings are made possible, inflected, 
and even negated by systems of conventions. To correct this problem, Culler 
wants us to widen our view using the model of linguistics, which would allow 
us to see beyond meanings to their conditions of possibility and intelligibility. 
The primary task of the discipline then, the activity formative of the disci- 
pline, would be the “attention to mechanisms of signification in texts and text- 
like situations” (“Future” 30), the description and exposition of codes. 

However, if we divorce poetics entirely from interpretation, it becomes 
difficult, even pointless, to say what a text’s effects are for. We lose any motiva- 
tion to articulate argumentatively the significance of a textual performance’s 
use of a signifying mechanism, since our focus has zoomed out, like linguistics, 
to “the underlying set of rules — the grammar” (Culler, Literary 229). The goal 
would become to make these rules explicit rather than to generate an argu- 
ment about how the text’s conscious or unconscious use of such rules signifi- 
cantly informs or deforms its possible and arguable meanings. The effect of 
foregrounding poetics at the expense of interpretation is that the text becomes 
at best the aperture onto “the mechanisms of linguistic and social interaction” 
(172). Think of the student papers that extensively treat a linguistic or ideolog- 
ical mechanism but cannot fold that discussion into one that articulates why it 
is important to talk about the mechanism in relation to this particular literary 
instantiation. If the instance is merely incidental, then the method of analysis 
and the object of exposition may as well belong to a different, more “socially 
relevant” field of study. 

Rather than raising up poetics as the answer to disciplinary crisis, it would 
be better to expand the hermeneutic orientation to include the concerns of 
poetics, as well as all the other approaches to the multiplicity of significances 
that always exceed the narrow concern for “the meaning.” In other words, 
instead of casting our critical activity as peering through an aperture, I pro- 
pose that we think of it as engaging the thickness of the signifying object, the 
dense interweaving of significances it comprises. This kind of reading means 
identifying interpretive problems and constructing arguments in a way that is 
open to the social and political significances inevitably projected by the history 
of language’s use, but refusing the easy pragmatism in which the play of sig- 
nificances condenses too quickly into simple meanings or expressions of forms 
of life. This expanded hermeneutics would turn E.D. Hirsch’s distinction 
between meaning and significance upside-down, since it is not interested in 
the validity of interpretation but in the multiplicity of arguable angles, which 
presuppose the multiplicity of significant resonances created by the intersec- 
tions of many different strands of many different discourses. A text’s message, 
and even a text’s disruption or refusal of meaning, is but one significant aspect. 

The interest in the thickness of the literary or cultural object as construct- 
ed by multiple readings (both iterative and methodologically divergent) retains 
the appreciation for both the formal and technical defamiliarization of refer- 
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ential language idealized in PcrlofF’s call for poetics. It also preserves Culler’s 
concern for the analysis of linguistic and historical structures that make the 
production of meanings possible. As an ideal outcome of a literary curricu- 
lum, the capacious practice of interpretation seeks to construct a view of the 
object comprising the broadest range of significances projected by the text. 
Such an excess of significances or interpretive aspects may come not only 
from formal devices and the manner in which they are wielded, but also from 
the situation of the reader, and from the encompassing context or historical 
itinerary in which reader and object cross paths. No source takes priority over 
the other: while each may generate different semantic weights and pressures 
within an argument, none can be taken as the sole ground of aspectuality, 
whether that source be language, intention, genres, devices, or history. 
Regardless of the provenance of significances, these aspects are activated by a 
basic, literary kind of activity — by posing interpretive questions and compos- 
ing a justification that gathers as many particular details of the signifying object 
as possible and articulates why they are important for understanding, experi- 
encing, or identifying with this particular, resonant organization of signifiers. 

IV 

If our recent MLA presidents are right about the present of our profes- 
sion, then the future of literary studies will depend on how well we invite stu- 
dents into this reading-seeing-writing practice at the heart of the literary cur- 
riculum. This practice might very well begin with Graff’s and Birkenstein’s 
textbook move, “They Say”/“I Say.” But while the composition of a basic case 
requires pushing against an alternate position, thick reading also requires lis- 
tening for the multiplicity of possible resonances and listening to the multiplic- 
ity of critical rationales that would give the “They say” and “I say” move its 
substance and stakes. By treating the curricularizcd classroom as the site of 
disciplinary formation, we invite students into a truly plural discourse commu- 
nity gathered around a distinctive and valuable mode of inquiry and response, 
a shared way of questioning and arguing, in addition to a provisional body of 
conflictual knowledge. It is assessment, I have been arguing, that can help us 
construct this expanded hermeneutic project out of our many collective 
desires, not only to know, but to experience, understand, perform, inhabit, and 
proprioceive. 

Discourse-based assessment would press us to generate capacious idealiza- 
tions that remain fluid and revisablc. These idealizations would impact not 
only our course outcomes and program goals, but also our disciplinary and 
professional self-projections. The thick reading that results from our openness 
to multiplicity and our argumentative engagement of difference would stand 
as a distinctive disciplinary activity. For example, consider the contrast between 
the literary reading produced by a truly curricularizcd program, on the one 
hand, and the lucidity and parsimony idealized in the sciences and social sci- 
ences, on the other. In the latter, logical structures must avoid contradictions. 
But in literary studies, while there are degrees of persuasiveness (itself subject 
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to argument) in negotiating contradictory aspects, contradiction does not nec- 
essarily disable or cancel out a critical reading. Rather, contradiction often 
forms the condition for a more capacious account, one that is less concerned 
to explain how the tension was produced by a structure than to build the sig- 
nificance of that tension (and even its production) into the dense and resonant 
dynamics of the object. 

Of course, literary studies, so conceived, could never become the star of 
the contemporary university. But neither would its identity collapse into histo- 
ry, sociology, or anthropology. And its differences from the other disciplines 
give it a unique role to play. That role would not involve duplicating the work 
of producing historical knowledges, but it would continue to see in the texture 
of signifying objects the epistemological and ethical work of historical witness 
and social critique. That role would not entail the mapping of codes or the 
exposition of signifying mechanisms without some sense of how their mean- 
ing-production can be articulated to their meaningful uses and projections. Its 
power would come less from producing transgressive knowledges and more 
from inculcating a supple, dialogical, and responsive way of seeing, under- 
standing, and experiencing, out of which transgression amasses its stakes. It 
would refigure the indignity of mere “service” work into a composition of the 
full complexity of the world, in which making judgments based exclusively on 
knowledge claims seem too narrow and mechanical a form of participation. 
And the role of literary studies would help us balance our proofs and explica- 
tions with an interrogative and aspectual orientation that can continue 
nonetheless to maintain our chastening desires to be the less deceived. 

If the “history of modernism and the New Criticism is inevitably a histo- 
ry of the rise of the modern university as well,” as Rainey and Louis Menand 
say in their introduction to The Cambridge History of Literary Criticism (8), then 
my argument here ultimately calls for a rethinking of the standard alignment 
between modernism and the New Criticism, and of at least one of the pur- 
poses of the modern university. For modernism, when faced with a receding 
Sea of Faith, did not simply turn to anti-scientistic, non-discursive truths. In 
the absence of “timeless universals,” modernism initiated the critical project 
whose aim was an active pcrspectivism in time. It was happy to give up truth 
for the critical and imaginative energies generated by its method of compre- 
hensive, even encyclopedic, constructivism. The lesson of modernism is there- 
fore not justly embodied in the New Critical mission, which, as the standard 
argument goes, was to protect timeless meaning from the impositions of cpis- 
temic culture by means of paradox, ambiguity, and its bracketing of the 
world. It is better epitomized by the cultivation of artistic, literary qualities of 
attention — the aspectual seeing typical of complex interpretation — whereby 
ambiguity is often far too frugal and placid to describe the engagement with 
significant objects. 

The university of the twenty-first century might thus be called to a pur- 
pose in excess of its commitment to professionalized knowledges, and literary 
studies might make use of assessment to facilitate this call. The commitment 
to knowledge, as WJ.T. Mitchell recently pointed out in his 2003 symposium 
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on “ Critical Inquiry in the Twenty-First Century,” can never be too well served, 
in light of the Iraq War that was pushed through by a government and a cul- 
ture “immune to persuasion, argument, reason, or even the flow of accurate 
information” (327). But the other purpose that the corporatized university 
might yet take up could be informed by the “medium” idealizations which 
attach to Mitchell’s “medium theory” — the idealizations that do not combat, 
negate, or revolutionize knowledge so much as they focus on the way that our 
objects of study, and we, actively generate orientations toward knowledge, 
compose forms of experience that justify reading and writing as powerfully 
orienting activities, and locate supple, analogical, aspectual seeing at the heart 
of inquiry itself. 1 *^ The bleak times that Pcrloff sees for our profession will be 
the result, not of our inability to offer “a body of knowledge,” “definable 
expertise,” or a stable method for evaluating texts (“Presidential Address” 

655), but of our inability to cast the multiplicity of our knowledges, skills, and 
methods as more than a loose bag of incompatible stuff. 

Assessment can help us to transform literary studies’ much-bemoaned 
grab-bag of critical practices into a strange and dynamic array, a resource that 
continues to render cultural products and moments as thick, striated, and 
internally differentiated. A fully curricularizcd discipline, in the sense outlined 
above, would maintain the historical form of seeing inspired by the cultural 
anthropology of Clifford Geertz and the New Historicism of Stephen 
Greenblatt. But its hermeneutic and argumentative core would steer it away 
from the sheer description and the hyperopic explanation of poetics, cultural 
or otherwise. As far as our institutional status goes, I hope literary studies’ 
“medium” project steers us between two fates: the situation of the early I. A. 
Richards, who, because he was not salaried by Magdalene College, had to col- 
lect money from his students weekly at the classroom door; and the total 
absorption of literature into high-bandwidth information dumps and knowl- 
edge work, a predicament allegorized by Alan Liu’s recent updating of the 
epilogue to Renaissance Self-Fashioning, in which the shot of Greenblatt reading 
Geertz on the plane is engulfed by the wider scene of “corporate intelli- 
gentsia” typing away on their laptops (Laws of Cool 4). 


NOTES 

I would like to thank my colleagues at Seattle University, especially John Bean, a master 
teacher whose devotion to teaching inspired not only this essay, but also the idea that I could 
send him multiple drafts. I am also grateful to Therese Huston and David Green of the 
Seattle University Center for Excellence in Teaching and Learning for their support. And 
thanks to Charles Altien, whose example (and feedback) always teaches me something about 
the value of justification. 

1 Pcrloff questions current practices of “interdisciplinarity” by asking about 
the identifiability of the disciplines between which an “inter-” might fall. 
She claims that “it would be more accurate to call the predominant activi- 
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ty of contemporary literary scholars other-disciplinary rather than interdisci- 
plinary” (“Presidential Address” 655; original emphasis). 

~ Deresiewicz examined the MLA’s 2007 Job Information List in his cranky 
online review of Gerald Graff’s twentieth-anniversary reprint of Professing 
Literature. For the exact numbers, see David Laurence, Natalia Lusin, and 
Stephen Olsen, “Report on Trends in the MLA Information List, 
September 2007.” In Table 6 of that report, 29.1% of jobs in 2006-07 
were indexed under “Composition and Rhetoric.” The next highest 
number was 28.2%, for “Literature, British Isles.” 

Q 

0 That tree has a branch called “Creative Writing.” In her column, subtitled 
‘“Creative Writing’ among the Disciplines,” Perloff found that the creative 
cousin of composition studies is also on the rise. Examining the 2006 Job 
Information List, Perloff found 103 positions in “Creative Writing and 
Journalism” but only 36 in twentieth-century literature (4). 

^ Sullivan ends his letter with a question: “What would happen if the MLA 
elected its presidents from an activist bottom rather than a prestigious 
top?” (255). Perloff responded: “Sullivan will be happy to hear that 
Gerald Graff, the 2008 MLA president, whose fields of specialization 
include composition theory and the pedagogy of English teaching, plans 
to focus on precisely the issues Sullivan has in mind” (256). To me, Graff’s 
MLA presidency signaled the profession’s need, or at least its opportunity, 
to rethink the discipline by exploring the relationship between “service” 
and profession, teaching and research. 

All postings on the MLA blog and the Inside Higher Ed website can be 
found at <http://www.mla. org/blog&topic= 121 > and 
<http://www.insidehighercd.com/views/2008/ 02/21/ graff> respectively. 
On the MLA blog, Graff’s original text seems to have been replaced by 
another article, but can be found in the MLA Newsletter 40.1 (2008): 3-4. 

® If the reader does have an interest, I am in the process of finishing a ped- 
agogy article called “From Close Reading to Thick Reading, Or, Is There 
A Curriculum In This Glass?” Here I describe the specific pedagogical 
and curricular adjustments we made to our program, as well as the prob- 
lems we are still discovering in the common curricular intuitions about the 
foundational status of historical surveys or close reading courses. 

^ “Patterned isolation” is a phrase from Laurence Veysey’s The Emergence of 
the American University (338), which Graff uses in his crusade against disci- 
plinary fragmentation and incoherence in Professing Literature. 

Scholes continues: “Which means that we need to evaluate scholarship as 
it manifests itself in teaching and not just in objects like published books” 
(8). This speaks to another face of the crisis of the discipline, the way in 
which the distinction between research and teaching feeds into the corpo- 
ratized university looking to cut “unproductive” faculty. John Guillory’s 
contribution to this 2005 forum in Profession calls for a “conversation about 
the evaluation of humanities scholarship,” so that we acknowledge that 
“increasing the quantity of scholarship cannot improve its quality” (28; 

33). Our methods of self-evaluation, Guillory argues, impacts the public 
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valuing of the humanities as a whole. 

^ A side benefit of this would be that, instead of thinking of our classes as 
representing our various specialized research interests, “we should think of 
our publications as another mode of teaching” (Scholes 8). Compare my 
curricular stance and its justifications with Charles Altieri’s description of 
the situation: “Literary studies is a discipline with a long history of critical 
idealization in search of authority, and literary critics have long and prob- 
lematic histories of taking pleasure and a sense of power from what they 
teach and explain. One might argue that the discipline has little else to 
offer except this personal satisfaction, since it tends to be too speculative 
for philosophical argument and too cynical or self-reflexive in its concern 
for the intricacy of particular texts to provide the kind of data and repre- 
sentative anecdotes most historians still desire” (Canons 5). 

lu In Altieri ’s account, the lack of stakes (of whether something works or 
doesn’t) led to an overemphasis on meaning, which at first was cast as the 
celebration of anti-scientific, non-discursive truths, then as themes organ- 
izing the totality of the structure, and finally as the socio-historical knowl- 
edge of contexts. In his resistance to “epistemically oriented 
Enlightenment values” (“Taking Lyrics Literally” 260), Altieri makes a 
case for an outcome that is not self-knowledge but rather the identifications 
a text makes available based not on content but performative energies. 

1 1 See Culler, Literary 9, as well as “The Luture of Criticism,” in which he 
characterizes earlier, pre-structuralist interdisciplinary work as “reduction- 
ist, ignoring complexities of literary language and making the text, in 
effect, a symptom, whose true meaning lay elsewhere” (29). See also 
Pcrloff’s “Crisis in the Humanities”: “Lor cultural criticism, the only real 
justification for literary study is the concession that poems and novels can 
do ‘cultural work.’ Lrom this perspective, a poetic text is primarily to be 
understood as a symptom of the larger culture to which it belongs and as 
an index to particular historical or cultural markers” (673). 

Mitchell was divided between asking thinkers to “sober up about the [rev- 
olutionary] expectations critical theory can realistically envision today” 
and calling for a turn to “medium theory,” which could best explore 
socio-epistemic conditions by its tactical position in a duree longer than the 
explosive instant, by flying below the explanatory ambitions of “high the- 
ory,” and by focusing on the mediations of our world in the full range of 
material forms of signification (332-34). Lor the editorial board members’ 
responses to the problem of the future of inquiry, see the entire issue of 
Critical Inquiry 30.2 (2004). 


WORKS CITED 

Altieri, Charles. Canons and Consequences: Reflections on the Ethical Force of 
Imaginative Ideals. Evanston: Northwestern UP, 1990. Print. 

. “Taking Lyrics Literally.” New Literary History 32 (2001): 259-81. Print. 


214 



THE FUTURE OF LITERARY CRITICISM: ASSESSMENT, THE CURRICULARIZED CLASSROOM, AND THICK READING 


Bean, John, David Carrithers, and Theresa Earenfight. “Transforming WAC 
through a Discourse-Based Approach to University Outcomes 
Assessment.” WAC Journal 16 (2005): 5-21. Print. 

Bennett, Michael. “Critique of the ‘liberal take’ on LOA.” Weblog comment. 
28 Feb. 2008. “Assessment Changes Everything.” Gerald Graff. Modern 
Language Association, n.d. Web. 15Jul. 2008. 

Carroll, Lee Ann. Rehearsing New Roles: How College Students Develop as Writers. 
Carbondale: Southern Illinois UP, 2002. Print. Studies in Writing and 
Rhetoric. 

Critical Inquiry 30.2 (2004): 225-492. Print. 

Culler, Jonathan. “The Future of Criticism.” The Current in Criticism: Essays on 
the Present and Future of Literary Theory. Ed. Clayton Koclbe and Virgil 
Lokke. West Lafayette: Purdue UP, 1987. 27-42. Print. 

. The Literary in Theory. Stanford: Stanford UP, 2007. Print. 

Deresiewicz, William. “Professing Literature in 2008.” The Nation.com. The 
Nation, 1 1 Mar. 2008. Web. 19 Aug. 2008. 

Fendrich, Laurie. “A Pedagogical Straitjacket.” Chronicle Review. Chronicle of 
Higher Education, 8 Jun. 2007. Web. 19 Aug. 2008. 

. “Gerald Graff’s Essay.” Weblog comment. 21 February 2008. 

“Assessment Changes Everything.” Gerald Graff. Inside Higher Ed, 21 Feb. 
2008. Web. 19 Aug. 2008. 

Foley, Barbara Clare. “Outcomes Assessment.” Weblog comment. 21 Feb. 
2008. “Assessment Changes Everything.” Gerald Graff. Modern Language 
Association, n.d. Web. 15 July 2008. 

Gallop, Jane. “The Historicization of Literary Studies and the Fate of Close 
Reading.” Profession (2007): 181-86. Print. 

Graff, Gerald. Clueless in Academe: How Schooling Obscures the Life of the Mind. 

New Haven: Yale UP, 2003. Print. 

. “The Future of Theory in the Teaching of Literature.” The Future of Literary 

Theory. Ed. Ralph Cohen. New York: Routlcdge, 1989. 250-67. Print. 

. “President’s Column: Assessment Changes Everything.” MLA Newsletter 

40.1 (2008): 3-4. Print. 

. Professing Literature: An Institutional History . Chicago: U of Chicago P, 1987. 

Print. 

Graff, Gerald and Cathy Birkenstcin. “They Say”/“I Say”: The Moves that Matter 
in Academic Writing. 2nd cd. New York: Norton, 2006. Print. 

Green, Daniel. “Literature Itself: The New Criticism and Aesthetic Experience.” 
Philosophy and Literature 27.1 (2003): 62-79. Print. 

Guillory, John. “Valuing the Humanities, Evaluating Scholarship.” Profession 
(2005): 28-38. Print. 

Hirsch, E.D. Validity in Interpretation. New Haven: Yale UP, 1967. Print. 
Jameson, Frcdric. “Symptoms of Theory or Symptoms for Theory?” Critical 
Inquiry 30.2 (2004): 403-08. Print. 

Laurence, David, Natalia Lusin, and Stephen Olsen. “Report on Trends in the 
MLA Job Information List, Sept. 2007.” Modern Language Association. MLA, 
Sept. 2007. Web. 19 Aug. 2008. 


215 



Linkon, Sherry Lee. “How Gan Assessment Work for Us?” Academe 9 1 .4 
(2005): n.pag. Web. 19 Aug. 2008. 

Liu, Alan. The Laws of Cool: Knowledge Work and the Culture of Information. 
Chicago: U of Chicago P, 2004. Print. 

Menand, Louis, and Lawrence Rainey. Introduction. The Cambridge History of 
Literary Criticism: Modernism and the New Criticism. Ed. A. Walton Litz, Louis 
Menand, and Lawrence Rainey. Vol. 7. Cambridge: Cambridge UP, 2000. 
1-16. Print. 

Mitchell, WJ.T. “Medium Theory: Preface to the 2003 Critical Inquiry 
Symposium.” Critical Inquiry 30.2 (2004): 324-35. Print. 

Moretti, Franco. “Conjectures on World Literature.” New Left Review 1 (2000): 
54-68. Print. 

Murray, Heather. “Close Reading, Closed Writing.” College English 53.2 (1991): 
195-208. Print. 

Pcrloff, Marjorie. “Crisis in the Humanities? Reconfiguring Literary Study for 
the Twenty-First Century.” Theory’s Empire: An Anthology of Dissent. Ed. 
Daphne Patai. New York: Columbia UP, 2005. 668-83. Print. 

. Fetter. PM I A. 123.1 (2008): 255-56. Print. 

. “Presidential Address 2006: It Must Change.” PMLA 122.3 (2007): 652- 

62. Print. 

. “President’s Column: ‘Creative Writing’ among the Disciplines.” A ILA 

Newsletter 38.1 (2006): 3-4. Print. 

Potter, Russell. “Assessment.” Weblog comment. 13 Feb. 2008. “Assessment 
Changes Everything.” Gerald Graff. Modern Language Association , n.d. Web. 
15 Jul. 2008. 

Rainey, Fawrence. Institutions of Modernism: Literary Elites and Public Culture. New 
Haven: Yale UP, 1999. Print. 

. Introduction. A Poem Containing History: Textual Studies in the Cantos. Ed. 

Fawrence Rainey. Ann Arbor: U of Michigan P, 1997. Print. 

Renker, Elizabeth. The Origins of American Literature Studies: An Institutional 
History. Cambridge: Cambridge UP, 2007. Print. 

Scholes, Robert. “Whither, or Wither, the Humanities?” Profession (2005): 7-9. Print. 
Sullivan, James. Fetter. PMLA. 123.1 (2008): 255. Print. 

Veysey, Faurence. The Emergence of the American University. Chicago: U of 
Chicago P, 1965. Print. 


BIOGRAPHY 

Charles M. Tung is Associate Professor of English at Seattle University, 
where he teaches modernism, literary theory, and Asian American literature. 
He is completing a book manuscript entitled “Reimagining the Present: 
Modernist Temporalities, Alternate Historicity, and Time Travel,” which con- 
nects modernism’s obsession with time to later cultural projects that imagine 
history and the present otherwise. 


216 



A PROGRESSIVE CASE FOR EDUCATIONAL 
STANDARDIZATION: HOW NOT TO RESPOND TO 
CALLS FOR COMMON STANDARDS 

GERALD GRAFF AND CATHY BIRKENSTEIN 


IN THE RESPONSES FROM HIGHER education to the “Spellings 
Report,’' the 2006 Commission report to Secretary of Education Margaret 
Spellings, A Test of Leadership: Charting the Future of U.S. Fligher Education , one 
particular argument was made over and over again: that educational standard- 
ization of the sort implicitly called for in the Report — and by others in the 
Standards Movement — is neither possible nor desirable. According to this 
argument, the standardization entailed by the Report’s recommendations 
would destroy what makes American colleges and universities great: their irre- 
ducible diversity, which can never be reduced to a common standard or meas- 
ure of educational effectiveness. And today, as the call for common standards 
that marked Spellings’ tenure under the Bush administration has now been 
embraced rather than rejected by President Obama’s Education Secretary 
Arne Duncan, who is calling for national “college readiness standards,” many 
in the education world continue to oppose common standards as a threat to 
educational diversity. 

This, we will argue, is the wrong way to respond to calls for common edu- 
cational standards, particularly in the realm of higher education. Although the 
“S” word has become virtually synonymous among most academics with ped- 
agogical sterility, we want to make a progressive case for educational standard- 
ization by pointing out its unappreciated democratic potential. 

The Spellings Report took American higher education to task for what it 
described as “a remarkable absence of accountability mechanisms” to ensure that 
colleges help educate as broad a spectrum of students as possible. Specifically, it 
charged that the nation’s colleges and universities had not done enough to 
align college and high school literacy instruction, leaving many high school 
students unprepared to go on to college. In addition, the Report charged, 
many students who do go on to college never complete their degrees, partly 
because college tuition is so expensive, but also partly because, in the Comm- 
ission’s words, “most colleges and universities don’t accept responsibility for 
making sure that those they admit actually succeed” (United States, x). 
Furthermore, the Report complained, “there are disturbing signs that many 
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students who do earn degrees have not actually mastered the reading, writing, 
and thinking skills” required by today’s competitive global knowledge econo- 
my, and that “the consequences of these problems are most severe for students 
from low-income families and for racial and ethnic minorities” (vii). 

The solution proposed by the Report was essentially to increase the free- 
market competition between colleges, heightening the competitive incentives 
presumably now lagging on campuses to improve the quality of undergradu- 
ate education while simultaneously cutting costs. To this end, the Report pro- 
posed that common standards be established for college-level work and that 
college students be tested to determine how much they are learning. Without 
such common standards and tests, the Report insisted, it would be difficult for 
high schools to prepare their students for college and for students, parents, and 
taxpayers to compare the quality of education offered by one college with that 
of another. Such consumers would be able to assess the return on their invest- 
ment only if they had a reliable measure of how well different institutions are 
preparing students to succeed outside college. 

A number of aspects of the Report did draw legitimate opposition from 
higher education. It was and still is legitimate, we think, to fear that the tests 
the Report proposed for higher education would end up resembling the intel- 
lectually dubious tests that still dominate American schools under the No 
Child Left Behind Act (NCLB). It is also legitimate to fear that the Report 
represented what Douglas C. Bennett, President of Earlham College, called 
an attempt “to improve higher education on the cheap” (1), that test results 
would be used to further defund already financially straightened colleges, that 
the common standards the Report calls for would be applied in draconian 
ways without faculty consent, and that these standards would be developed 
by corporate managers and public officials with little knowledge of academic 
culture. Finally, it is legitimate, in our view, to be concerned about the free- 
market ideology underlying the Report and its narrowly vocational vision of 
higher education. 

But opposing the Spellings Report on the grounds that American colleges 
are too diverse to be judged by any common standard strikes us as unduly 
defensive and unreflective. Over and over Spellings’ critics insisted that any 
attempt to apply such a common standard to colleges will inevitably result in a 
“one-size-fits-all” straitjacket that will destroy what is most distinctive about 
our institutions and the heterogeneous student populations that they serve. 
Thus in its response to the Report, the American Association of University 
Professors complained that the Commission seemed oblivious to the harm that 
its “call for standardization . . . would inflict on the diverse missions of our col- 
leges and universities” (4). In another response to the Report, Ronald Crutcher, 
the President of Wheaton College of Massachusetts, asserted that “it would be 
an enormous mistake to measure each institution by the same yardstick,” since 
“research universities, community colleges, public institutions and private lib- 
eral arts colleges have different missions and serve different populations” (2-3). 
President Bennett of Earlham complained that 
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the commission comes dangerously close to implying that a one- 
size-fits-all measure should be used. The diversity of our institu- 
tions’ missions and our students calls for a diversity of meas- 
ures — not some Washington-imposed single test. (1) 

Along similar lines, John Churchill, the Secretary of the Phi Beta Kappa 
Society, insisted that the Report’s ‘’demand for common measurement” threat- 
ens what has been the strength of American higher education, its “diversity” 
and “decentralization” (2). And Jill Beck, the President of Lawrence College, 
argued that we should resist ‘“one size fits all’ test instruments,” since “a fun- 
damental strength of higher education is its remarkable institutional diversity.” 
The “Commission’s ... misguided benchmarks,” Beck continued, “have the 
effect of trying to homogenize American higher education” (3). 

We see four major problems with this anti-standardization position. First, 
the wholesale rejection of common standards fails to distinguish between good 
and bad forms of standardization. The standardized tests that characterize 
NCLB (following a long history of assembly-line approaches to schooling) 
have given standardization such a bad name that it has become too easy to 
reject standardization as such through a sort of guilt by association. As long as 
we equate all standardization with invidious, NCLB-style testing or the 
McDonaldization of American culture, we ignore the existence of other forms 
of standardization — environmental, health, and safety standards, to mention 
only a few obvious examples — that most of us readily accept or insist on. In 
the wake of the 2010 British Petroleum oil disaster in the Gulf of Mexico, it is 
hard not to see national and even international standards as a good thing for 
the cause of justice and human welfare. 

Second, the blanket rejection of educational standardization is undemoc- 
ratic. To say that academic competence can’t be judged by any standardized 
measure mystifies such competence by turning it into a matter of taste or 
whim — an ineffable je ne sais quoi mysteriously possessed by a minority of supe- 
rior talents — rather than a set of practices that can be identified, modeled, 
and made generally accessible. It’s a short step from telling the Spellings 
Commission, “Sorry, but we colleges are just too diverse to be measured by 
any common standard,” to telling students, “Sorry, but the basic skills that you 
need to succeed in college are just too complex and heterogeneous to be 
explained to you clearly.” 

Third, attacks on educational standardization simply mirror and reinforce 
American education’s disconnected, fragmented status quo. American colleges 
today can indeed be proud of their impressive intellectual and disciplinary 
diversity. What is far less impressive, however — and here we agree with 
Spellings — is their record in helping students negotiate that diversity by pro- 
viding them with the skills needed to make sense of it. Given the discontinu- 
ities of the educational system (discontinuities that standardization would help 
counteract), students have no assurance that what they learn in one grade 
level, institution, discipline, or course will be recognized, rewarded, and built 
on in the next. A minority of high achievers manages to see through the cur- 
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riciilar disconnection to detect the fundamental critical thinking skills that 
underlie effective academic work in any course or discipline. The majority, 
however, must resort to the familiar tactic of giving each successive instructor 
whatever he or she seems to want and then doing that again with the next 
instructor and the next. For these students, giving instructors whatever they 
want — assuming students can figure out what that is in the first place — 
replaces cumulative socialization into academic ways of thinking and writing. 
College thus becomes a sequence of disciplines and courses, each tending to 
present a different picture of what academic work looks like, but few having 
the overarching status that a more standardized curriculum would confer. 

Even when some courses do have such an overarching status, as in first-year 
composition and introductory courses, there is often little uniformity between 
such courses, or instructors of other courses don’t refer to these courses or 
even know what is going on in them. 

Last and most important, it is simply not true, as the anti-standardization 
argument has it, that colleges are so diverse that they share no common stan- 
dards. Just because two people, for instance, don’t share an interest in baseball 
or cooking, it does not follow that they don’t have other things in common. 
Nor does it follow that, because several colleges have different types of facul- 
ties or serve different student populations, they can share no common peda- 
gogical goals. A marketing instructor at a community college, a Biblical studies 
instructor at a church-affiliatcd college, and a feminist literature instructor at 
an Ivy League research university would presumably differ radically in their 
disciplinary expertise, their intellectual outlooks, and the students they teach, 
but it would be surprising if there were not a great deal of common ground in 
what they regard as acceptable college-level work. At the end of the day, these 
instructors would probably agree — or should agree — that collcge-cducatcd stu- 
dents, regardless of their background or major, should be critical thinkers, 
meaning that, at a minimum, they should be able to read a college-level text, 
offer a pertinent summary of its central claim, and a relevant response, whether 
to agree with it, to complicate its claims, or to offer a critique. Furthermore, 
though these instructors might expect students at different types of institutions 
to carry out these skills with varying degrees of sophistication, they would still 
probably agree that any institution that persisted in graduating large numbers 
of students deficient in these fundamental critical thinking skills should be 
asked to regroup and figure out how to do its job better. 

Spellings’ critics insisted, however, that such apparent agreement is illuso- 
ry. Thus in his response to the Report, Lee Shulman, then-President of the 
Carnegie Foundation for the Advancement of Teaching, argued that though 
educators may seem to agree on the importance of critical thinking as a stan- 
dard for college-level work, the term is used to mean so many different things 
that its usefulness as a standard is undermined. As Shulman put it, “common 
educational goals like ‘critical thinking’ . . . are often invoked for quite different 
achievements No single set of measures can do justice to all those varia- 

tions” (1). Like Spellings’ other critics mentioned above, Shulman demanded 
that colleges be free to take “many different approaches to higher education” 
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rather than being forced to converge “on the ‘one best system’” (1). 

But we need go no further than Shulman’s own prose — or that of other 
Spellings critics — for an example of the critical thinking skills whose common- 
ality he denied. Even as Shulman claimed that the concept of critical thinking 
is hopelessly diffuse, his writing, like that of Spellings’ other critics, showed that 
it involves such basic moves as: 

• Locating a controversial issue or problem; 

• Accurately summarizing the views of others; 

• Framing and explaining quotations; 

• Offering one’s own argument and explaining why it matters; 

• Moving between one’s own position and the one being 
responded to without confusing readers; 

• Weighing evidence; 

• Anticipating counterarguments; 

• Drawing conclusions. 

These fundamentals — whose ubiquity in the intellectual world Shulman 
denies — are precisely those that most students fail to learn. And in our view, 
students will go on failing to learn these fundamentals unless they are stan- 
dardized across all domains and levels — that is, are represented with enough 
redundancy, consistency, and transparency that students can recognize them as 
fundamentals rather than as one instructor’s arbitrary preferences competing 
for their attention among many. 

In sum, then, there has to be a better way to respond to the call for com- 
mon standards in higher education than defensively insisting on our irreducible 
diversity. Instead, we in higher education should be opening up debates on 
campuses across the country over whether there are common practices that 
underlie that diversity. If a consensus emerges that there are, as with good 
leadership we think it will, we should then work collectively — with the full par- 
ticipation of college faculties — to identify and standardize those practices so 
that students can more readily acquire them. Engaging in this standardization 
process is important, we think, not just because, without it, NCLB-style ver- 
sions of standardization may be imposed on us unwillingly, but because intelli- 
gent standardization is critical to our mission of democratic education, which 
entails being as explicit as possible about the key moves of academic success, 
and helping as many students as we can to master them. In our view, higher 
education does need common standards, even if some of those calling for the 
standards have a political agenda many of us disagree with. 

But how can the fundamental moves of critical thinking be standardized: 
that is, represented with enough consistency and redundancy across the cur- 
riculum that all students — not just the elite few — can see these moves as fun- 
damental? And how can this be done in a way that allows the educational 
results to be assessed and measured? 

The first step, in our view, is to identify and name these fundamentals in 
terms that are simple and familiar enough to be grasped and retained by the 
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vast majority of students as they move from course to course, but comprehen- 
sive enough to do justice to the complexity and heterogeneity of academic 
practices. Our candidate for such a formulation, as we have already suggested, 
is the practice of summary and response. On the one hand, summarizing and 
responding is a familiar argumentative skill that students have practiced virtu- 
ally every day since childhood (e.g, “But you said if I cleaned up my room 
tonight I could go out with my friends.”). As we see it, summary and response 
gets as close as any formulation can to the universally human call-and- 
response practice of making claims not out of the blue, but as responses to 
others. On the other hand, summarizing and responding encompasses all of 
the most advanced academic skills, including (in addition to those listed earli- 
er) close reading, interpretation, and analysis, working with factual, statistical, 
and textual forms of evidence, and even the ethical ability to entertain oppos- 
ing perspectives, putting ourselves in the shoes of those who disagree with us. 
And though this summary-response practice is deployed in different ways in 
different academic disciplines, there is no discipline that does not require that 
we enter a conversation, stating our views not in a vacuum but (as Shulman’s 
comments demonstrated above) as a response to what others in the field have 
said or might say. 

Describing this transdisciplinary practice in more polemical terms, the influ- 
ential rhetoric and composition specialist, David Bartholomae, observes that 

the best student writing works against a conventional point of 

view The more successful writers set themselves ... against 

what they defined as some more naive way of talking about their 
subject — against “those who think that ...” — or against earlier, 
more naive versions of themselves — “once I thought that.” (641) 

If this view is right — that the best student writing engages (challenges or adds 
to) other perspectives — then why withhold this crucial information from stu- 
dents? Why not be explicit about this key to academic success? 

But this first step — highlighting summary-response across the grades, dis- 
ciplines, and courses — is not enough. A second step is needed in which we go 
beyond simply explaining that responding to others is the central move of aca- 
demic culture, and provide training devices — concrete templates or scaffold- 
ings — that enable students to enact this move in their writing. 

Bartholomae provides an example of such a training heuristic when he 
recalls one of his undergraduate teachers once suggesting to him that, when- 
ever he was stuck for something to say in his writing, he try out the following 
“machine”: 

While most readers of have said , a 

close and careful reading shows that . (641) 

For students unsure about the basic shape of academic discourse, a scaffolding 
like this could help them make a claim and indicate why that claim matters by 
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showing what alternate claim it is correcting, supplementing, complicating, or 
otherwise is in dialogue with. 

Following Bartholomae’s lead, we have published a textbook, “They 
Say”/“I Say”, in which we provide templates like the following that prompt stu- 
dents to engage dialectically with the views of other thinkers and writers: 

In recent discussions of , a controversial issue 

has been whether . On the one hand, some argue 

that . From this perspective, . On the 

other hand, however, others argue that . In the 

words of one of this view’s main proponents, “ 

According to this view, . In sum, then, the issue is 

whether or . 

My own view is that . Though I concede that 

, I still maintain that . For example, 

. Though some might object that , I 

reply that . The issue is important because 


At first glance, it is easy to dismiss such formulaic devices for being too 
mechanical and prescriptive. But these complaints ignore the fact that such 
models are open to improvisation. These complaints also ignore the fact that, 
while experienced writers unconsciously absorb models like this through their 
reading, most students do not. Most students will never make a move like “My 

point is not , but ,” or even “I agree (or disagree) 

because ” unless given explicit prompts for doing so. 

Indeed, there is even reason to believe that it is not just humble under- 
graduates, but graduate students and faculty members as well, who need 
explicit help making these key academic moves. In a textbook addressed to 
graduate student writers across the academic disciplines, John Swales and 
Christine Feak explain that to establish the importance of their own claims, 
writers must “indicate the gap in previous research” by “reviewing previous 
research” (244). In a textbook addressed to thesis and dissertation writers, Irene 
L. Clark offers the following formulas for entering academic conversations: 

Some scholars who write about this topic say . Other 

scholars who write about this topic disagree. They say . 

My own idea about this topic is . (20-21) 

Along similar lines, the National Academy of Education requires applicants 
for its postdoctoral fellowship to complete the following template in fifty words 
or fewer: “Most scholars in the field now believe ...; as a result of my study 
...” (National Academy of Education). 

Likewise, the editors of the leading science journal Nature feel obliged to 
provide prospective contributors with writing guidelines that follow a classic 
summary/ response format, requiring that all submissions open with a clear 
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declaration not just of the authors’ central findings, but of how those findings 
compare with “previous knowledge” (Nature Publishing Group). If, as these 
examples suggest, even those at the highest reaches of academe need explicit 
help making the standard moves of academic critical literacy, think how much 
more struggling undergraduate and high school students need it. 

Dialectical templates like these can also help meet one of the key chal- 
lenges of outcomes assessment: avoiding what might be called the laundry list 
trap, in which so many different assessment criteria are offered that assessment 
ends up mirroring the fragmented academic curriculum itself, so overwhelm- 
ing students that they come away with no solid grasp of academic literacy’s 
basic shape. This trap can be avoided by developing exit examinations — and 
gearing courses and programs around them — that ask students to enter the 
academic conversation. For example, at the completion of any course or pro- 
gram of study, students could be asked to compose exit essays, using templates 
like the following that reveal the extent to which they have learned to frame 
and then enter a conversation in their field: 

Before I began my major in , I, probably like most 

people, assumed that . But having studied the field, 

I now see that it’s far more complicated, primarily because 

In contrast to some researchers in the field of 

who suggest , other researchers suggest 

. My own view is that . 

Though assessment templates like these might look disarmingly simple, com- 
pleting them would actually require students to command a full range of aca- 
demic competencies, from demonstrating familiarity with basic information in 
the field and knowledge of its key terms, concepts, and controversies to the 
ability to manage basic writing mechanics. 

Is this, then, a one-size-fits-all approach? Yes. And that’s precisely why we 
think it has a chance to work, especially if it can be implemented democrati- 
cally, with a high degree of faculty buy-in. For the more we proliferate multi- 
ple objectives and standards, the less chance there is that large numbers of 
students — or teachers, for that matter — will assimilate any of them. Conversely, 
the more we standardize — that is, collectively streamline, simplify, and rein- 
force — what it is we want students to learn the more chance we have of mak- 
ing academic critical literacy available on a mass democratic scale. 

A version of this essay was originally published, in the May-June 2008 issue of Academe. 
It can be found online at <http:/ /www.aaup.org/AAUP/pubsres/ academe/2008/ 
MJ /Feat/ graf.htm>. 


224 



A PROGRESSIVE CASE FOR EDUCATIONAL STANDARDIZATION: HOW NOT TO RESPOND TO CALLS FOR COMMON STANDARDS 


WORKS CITED 

American Association of University Professors (AAUP). “AAUP Statement on 
Spellings Commission Report.” American Association of University Professors. 
AAUP, n.d. Web. Sept. 2007. 

Bartholomae, David. “Inventing the University.” Cross-Talk in Comp Theory, A 
Reader. 2nd cd. Ed. Victor Villanueva. Urbana: NOTE, 2003. 623-53. 
Print. 

Beck, Jill. “Issues and Trends in Higher Education: Liberal Arts College 
Presidents Speak Out on Spellings Commission Report.” College News. 
Annapolis Group, 26 Sept. 2006. Web. Sept. 2007. 

Bennett, Douglas C. “Underinvesting in the Future.” Chronicle Review. 

Chronicle of Higher Education, 1 Sept. 2006. Web. Oct. 2006. 

Churchill, John. “In the Future of Higher Education, Will the Humanities 
Still Matter?” The Key Reporter 72 (2007): 2-3. Print. 

Clark, Irene. Writing the Successful Thesis and Dissertation: Entering the Conversation. 

Upper Saddle River: Prentice, 2007. Print. 

Crutcher, Ronald. “Issues and Trends in Higher Education: Liberal Arts 
College Presidents Speak Out on Spellings Commission Report.” College 
News. Annapolis Group, 26 Sept. 2006. Web. Sept. 2007. 

Graff, Gerald and Cathy Birkenstein. “They Say’V“I Say”: The Moves that Matter 
in Academic Writing. 2nd cd. New York: Norton, 2006. Print. 

National Academy of Education. “201 1 National Academy of Education/ 
Spencer Postdoctoral Fellowship Program.” National Academy of Education. 
NAEd, Jul. 2010. Web. Oct. 2010. 

Nature Publishing Group. “How to construct a Nature summary paragraph.” 
Nature. Nature Publishing Group, n.d. Web. Mar. 2006. 

Shulman, Lee. “A Response to the Final Report of the Commission on the 
Future of Higher Education.” Carnegie Fowidationfor the Advancement of 
Teaching. Carnegie Foundation for the Advancement of Teaching, 1 Sept. 
2006. Web. Sept. 2007 

Swales John M. and Christine B. Feak. Academic Writing for Graduate Students: 
Essential Tasks and Skills. Ann Arbor: U of Michigan P, 1 994. 

United States. Dept, of Education. Secretary of Education’s Commission on 
the Future of Higher Education. A Test of Leadership: Charting the Future of 
U.S. Higher Education. Washington: US Department of Education, 2006. 
ED.gov. Web. 5 Sept. 2010. 


BIOGRAPHIES 

Gerald Graff has been teaching since taking his PhD at Stanford University 
in 1963, first at the University of New Mexico, subsequently at Northwestern 
University, the University of Chicago, and currently at the University of 
Illinois at Chicago. He is the author, among other books, of Professing Literature: 
An Institutional History (U of Chicago P, 1987), Clueless in Academe: How Schoolmg 


225 



Obscures the Life of the Mind (Yale UP, 2003), and with Cathy Birkenstein, “They 
Say”/“I Say”: The Moves That Matter in Academic Writing (Norton, 2007). 
Especially known for his arguments on behalf of “teaching the conflicts,” 
Gerald has been a Guggenheim Fellow and a research fellow at the Institute 
for Advanced Study in the Behavioral Sciences at Stanford. In 2008, he served 
as President of the Modern Language Association of America, and in 2010 he 
received the MLA’s Francis Andrew March Award for Distinguished Service to 
the Profession of English. 

Cathy Birkenstein is the co-author with Gerald Graff of the textbook 
“They Say” /“I Say”: The Moves that Matter in Academic Writing (Norton, 2007) and 
has published essays on writing in College English and, with Gerald, in The 
Chronicle of Higher Education , College Composition and Communication , and Academe. 
She is now at work on a study of common misunderstandings surrounding 
academic discourse that grows out of her recent College English essay, “We Got 
the Wrong Gal: Rethinking the ‘Bad’ Academic Writing of Judith Butler.” 

She is a lecturer in English at the University of Illinois at Chicago, where she 
also co-directs the Writing in the Disciplines program. She has given lectures 
and workshops with Gerald at numerous colleges. 


226 



ENGLISH DEPARTMENTS, ASSESSMENT, AND 
ORGANIZATIONAL LEARNING 

DAVID MAZELLA 


How I Found Myself Writing an Essay about Assessment 

My experience with assessment is probably different from that of most 
people in English departments, because I learned about it through my involve- 
ment with my institution’s faculty senate. Because of these associations with 
the senate, I now view assessment as a matter of faculty governance, see it as 
necessarily multidisciplinary and collaborative, and regard it as inextricably 
linked with an ethos of institutional self-improvement. Moreover, because my 
discussions of assessment took place in the senate rather than in a particular 
department, assessment became for me something that faculty did , not some- 
thing that was done to them. All these factors helped me develop a more 
active, faculty-centered perspective on assessment than that held by many of 
my colleagues, who prefer to remain uninvolved with the assessments going on 
around them. 

I hope that this essay might demonstrate to those faculty members still 
skeptical about assessment how this activity can advance learning throughout 
the university, first of all by attending to and potentially enriching the interac- 
tions of faculty with students, but also by examining and illuminating other 
forms of interaction occurring throughout the university and beyond. 
Assessment has the power to deliver this kind of illumination, I would argue, 
because it represents an important form of both scholarly inquiry and scholar- 
ly communication, one that offers a view of student learning in the aggregate, 
and allows insights into the nature and quality of the interactions taking place 
between students and all the other groups involved with instruction and learn- 
ing on campus. 

All these aspects of assessment are on view in the case history I will offer 
about its role in the creation of a Center for Teaching Excellence (CTE) at my 
home institution, the University of Houston (UH). What I hope this case his- 
tory will develop is a different view of assessment, one that is faculty-driven, 
scholarship- and inquiry-based, committed to the improvement of learning, 
communications, and decision-making at an individual, departmental, and 
institutional level, and powerful enough to affect even our disciplinary self- 
understandings. By viewing assessment as intrinsically collaborative and schol- 
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arly, while having as its core function the encouragement of what is known as 
“organizational learning,” I hope to show that assessment also helps faculty 
steer their departments and universities, helps them internally align the vari- 
ous parts of the university around student learning, and assists universities in 
adapting to the exigencies of historical, political, and economic change. 
Directing and using the results of such assessments then becomes crucial to 
the development of a new, more self-regulated responsiveness within the uni- 
versity to its publics, and helps faculty and their advocates gather better evi- 
dence and devise stronger arguments for the support of higher education. 

My own learning about assessment began with my heavy involvement (as 
chair of the senate’s educational policy committee) with our Southern 
Association of Colleges and Schools (SACS) accreditation process, particularly 
in our Quality Enhancement Program (QEP) targeting undergraduate 
research, and working with scores of other faculty on a steering committee 
that helped select a topic and draw up a university-level QEP proposal. 1 My 
involvement continued when I co-created and taught a collaborative QEP 
grant-supported course and joined a QEP Assessment Task Force that has 
been working to measure the impact of our QEP efforts since 2008. 2 

Working on the QEP was pivotal for me because it forced me to articulate 
and refine the views about teaching I had held for over fifteen years, but it also 
made me confront a question I had never considered before, the notion of 
really assessing the impact of what I was doing in the classroom. As with 
many other accreditation activities, the SACS QEP grant programs are very 
straightforward about the need for grant-supported courses to document how 
they differ from existing courses and how they will measure the impact of 
their innovations on student learning (Commission on Colleges 21). This sort 
of demand was utterly new to me, and, like most other literature faculty, I had 
no idea how I might document or measure the effects of my teaching or 
course design. So I began my work on the QEP, as both an instructor and as 
one of the faculty helping to manage the program, with a thoroughly naive 
view of assessment. 

The teaching assessments I had previously experienced had not helped 
me see their connection to learning, either. Until my involvement with the 
QEP, assessment was mostly about the brown envelope I received at the begin- 
ning of every semester, which was filled with the previous semester’s student 
evaluations. This envelope held my students’ handwritten comments, along 
with some numbers I otherwise paid very little attention to. I always enjoyed 
the good comments, and resolved to do better in response to the not-so-good 
comments, but the whole enterprise seemed entirely disconnected from what I 
was trying to accomplish with my students. 

My real learning about assessment, however, began when the faculty senate 
leadership, myself included, decided that our university’s drive towards “tier- 
one” (meaning nationally recognized research university) status had to improve 
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not just our research profile but our undergraduate teaching, so that these were 
consonant with our understanding of a tier-one undergraduate experience. 

Our aspiration to be recognized as a research university probably requires 
some additional context to be fully appreciated: the University of Houston is a 
relatively young, urban, public university that anchors one of the fastest grow- 
ing regions in the country, and which features one of the most diverse student 
bodies found in any American public university Our mission has always been 
strongly defined by the needs of the city’s working- and middle-class students, 
even as the city and the university have grown together since the 1970s. Provi- 
ding a wide range of students access to higher education, including graduate 
and professional education, has always been balanced with our expanding 
research mission, particularly when Texas has long lagged behind other states 
in educational attainment, in terms of its high school, college, and advanced 
degree attainment rates. 3 

The state acknowledged as much when the Texas legislature passed a plan 
last year to create a special fund that would elevate one or more of the state’s 
emerging research universities to tier-one status, a plan that included bench- 
marks for both research expenditures and student success indicators (e.g., 
raised admissions standards and graduation rates) (Kever). Both our new presi- 
dent and the senate leadership agreed that UH needed to start its long climb 
toward tier-one status right away. They also understood that this effort would 
require significant faculty participation to improve both the research profile 
and the undergraduate experience. What gave our efforts extra urgency was 
the fact that the state had structured the tier-one process competitively, so that 
aspiring institutions would have to outperform one another if they wanted to 
get the largest portion of the quite limited pool of funds. Improvements would 
have to happen sooner rather than later, and at a pace set not by us but by the 
state’s competitive process. The state clearly wanted to see dramatic gains as 
quickly as possible among the largest possible number of institutions, without 
ever committing funds to any particular institution’s rise to tier-one status. 

Apart from the immediate impetus of the state’s new plan, another factor 
motivating the faculty was a long-held view that our institutional performance 
and research identity had grown out of sync with our local reputation, which 
remained that of a large commuter school in a city featuring one of the top 
private universities in the country (Rice University). It is also worth mention- 
ing that the city’s elites are for the most part graduates of the two existing tier- 
one public universities in Texas, the University of Texas at Austin and Texas 
A&M. One of the greatest sources of the faculty’s dissatisfaction was our 
fourth-tier designation in the US News and World Report (USNWRj college rank- 
ings, a designation belied by our research expenditures and recognized gradu- 
ate and professional programs. This designation resulted instead largely from 
undergraduate indicators like admissions selectivity, graduation rates, average 
class size, and alumni giving, all of which fall in line with our commuter 
school profile. 

It was this aspect of our academic reputation, the undergraduate experi- 
ence, which needed to be addressed in a comprehensive way. For this reason, 
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the senate leadership consistently argued to the faculty, regents, and local 
politicians that our drive to tier-one status represented not an abandonment, 
but an expansion of the school’s historical identity and mission. The tier-one 
drive, moreover, announced to all these constituencies that we now play a very 
different role in the region than we did when the school entered the state pub- 
lic system in 1963. And yet our location within a metropolitan area still experi- 
encing growth meant that our research and teaching missions would always be 
intimately connected with the expansion of the city and region in which we 
were embedded. 

For all these reasons, the senate leadership decided that, given our posi- 
tioning within the state’s economy and politics, we should publicly announce 
and formalize the evolving relation of our teaching to our research mission 
with the establishment of a new center for teaching excellence, which would 
be run by faculty and based in the senate. Such centers, we knew, were now 
common elsewhere, but they had never been successfully instituted at UH, 
because they had previously been treated as largely administrative initiatives 
with relatively little consultation from the faculty. We were determined this 
time around to make the input and contributions from the faculty the deciding 
factor for the creation of this new center. 

The center, or GTE, as we called it, would become part of the senate’s 
own program for institutional self-improvement, one that grounded discussions 
of faculty instruction and student learning around scholarly and collegial 
notions like evidence-based arguments, peer review, and best practices, which 
had not always been the case before (Mentkowski and Loacker 83). In this 
respect, the senate, far from playing a vestigial or symbolic role in governance, 
instead played a crucial and mediating role as an informal professional forum 
for voicing faculty concerns about teaching and instruction outside the usual 
territorial discussions occurring within departments (Birnbaum, “Latent 
Organizational Functions” 424). Such a mediating role would allow the CTE 
to become a “safe place” for discussion in which insights concerning instruc- 
tion on campus could be sifted, gathered together, and redirected toward par- 
ticular units for the purposes of improvement (Alpcrt 277). 

Working on this project gave every member of the GTE steering committee, 
including myself, a profoundly different view of assessment, because it offered 
each of us a campus-wide perspective on teaching and instruction. My back- 
ground as a teacher of eighteenth-century literature, or as a member of the 
English department, for example, was not enough for me to answer such large- 
scale questions by myself; yet my experiences as a teacher of literature turned out 
to be crucial for my collaborations with other faculty from fields like business, biolo- 
gy, and health and human performance. Our visits with faculty across the univer- 
sity made us ask ourselves: how successfully were we instructing our students as a 
university , and where did we need to direct our efforts and our resources? 

Assessment, in other words, was looking more and more like the inquiry I 
was encouraging in my classroom, with all the conditions and caveats that 
such a definition would entail. Thus, if assessment truly represented a form of 
inquiry, it would have to be self-directed and -regulated, and motivated by the 
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learner’s own needs, interests, and curiosity, while remaining responsive to the 
demands represented by existing scholarship, peer review, and collaboration 
(Zuckerman et al. 202-3). When viewed as scholarly inquiry, assessment 
becomes a way of continually extending one’s own (and the institution’s) 
knowledge about teaching and learning, both in- and outside one’s own class- 
room. But who, precisely, was the “learner” driving these inquiries and pursu- 
ing these questions? And how successful would these inquiries be if they 
remained strictly solitary pursuits? 

As I began to collect information and insights about instruction and cur- 
riculum from all over campus, I realized that there was a second and yet more 
dilhcult dimension to the job facing me and the group, which involved not just 
learning about conditions of teaching and learning, but in communicating our 
insights as broadly as possible. Only then would the university learn as an 
organization how to do its work better. Thus, if we wished to boost the universi- 
ty’s academic reputation, the CTE steering committee and its various projects 
would have to help the university act upon this knowledge and confront a 
number of longstanding problems in undergraduate instruction. And this new 
focus on university-wide “organizational learning,” to use the terms of Argyris 
and Schon, gave me a different view of the workings of my department 
(Kezar 99-104). So a major part of the challenge became setting aside an indi- 
vidualistic notion of inquiry and research, and thinking instead about broad- 
ening the process of learning into a more collaborative form of inquiry. 

What this shift in perspective taught me was that in order for assessment 
to serve learning at this broader, institution-wide level, it must be driven, first 
of all, by the questions of those most directly involved with instruction, to give 
its inquiries purpose, direction, and potential usefulness for action. At the 
same time, assessment must also communicate its findings to the responsible 
parties to help create a view of learning in the aggregate. 

I initially arrived at these insights about the need for organizational learn- 
ing when I encountered a number of knowledgeable people in different depart- 
ments and offices whose local experience and expertise seemed utterly absent 
from any higher-level discussions of instruction. It soon became clear to me, 
however, that any genuine and lasting improvements would have to come from 
what Argyris and Schon have termed “double loop” or “deep learning,” which 
allows organizations to engage in better collective deliberations while arriving 
at better decisions and more effective actions (Smith). If we accept Argyris’ def- 
inition of learning as the “detection and correction of error,” then “single- 
loop” learning is when organizations learn how to avoid certain kinds of pre- 
dictable errors, or how to apply their existing knowledge a little bit better: this 
kind of learning is incremental and essentially unrcflective in its behavior (206). 

In contrast, Argyris’ notion of “double-loop” learning is consciously recur- 
sive, self-reflective, and oriented toward problem-solving, decision-making, and 
collective action: it is “when errors are corrected by changing the governing 
values and then the actions” (206). To use Argyris’ most famous metaphor, “a 
thermostat is double-loop learning if it questions why it is programmed to 
measure temperature, and then adjusts the temperature itself” (206). As a the- 
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ory of the relation of learning to what Argyris terms “antilearning,” double- 
loop learning seems particularly helpful for explaining the difficulties faced by 
universities in acting upon the disciplinary knowledge their faculty are other- 
wise gathering and disseminating all the time. 

As one scholar of higher education muses in an article about the organi- 
zational difficulties of universities, “Does the [fiscal retrenchment of higher 
education] call for ‘single-loop’ organizational learning, that is, retaining the 
existing norms, goals, and structures and doing better the things we are now 
doing? Or does it call for ‘double-loop’ learning, that is, reformulating the 
norms, goals, and structures and embarking in innovative directions to create 
acceptable outcomes?” (Alpert 242-43). And the answer seems to be that in 
periods of severe historical crisis and rapid political, economic, and demo- 
graphic change, only organizations capable of substantively adapting to their 
changing circumstances — that is, only organizations capable of double-loop 
learning — will be to able to survive. 

Organizational learning involves, however, more than just a “detection 
and correction of error”; it also entails a self-governing, self-reflexive 
process of collective deliberation designed to bring to light all the unexam- 
ined assumptions (i.e., Argyris’ “theories in use”), excluded viewpoints, and 
alternative solutions that individuals and organizations would rather not 
confront. It represents a conscious, continuous strategy to minimize or 
interrupt the kind of defensive reasoning that allows people to avoid con- 
fronting the most embarrassing challenges to their views (i.e., what Argyris 
calls “disconfirmations”). If his prescriptions for double-loop learning are 
genuinely pursued, Argyris asserts, “defensive routines that are counterpro- 
ductive to learning are minimized, and genuine learning is facilitated. 
Embarrassment and threat are not bypassed and covered up; they are 
engaged” (215). 

As a scholar of the historical Enlightenment and its literature, I should 
add that I am fascinated by the absolute certainty of Argyris’ tone regarding 
the possibilities of a fully conscious “productive reasoning,” which seems to 
recapitulate the most optimistic hopes of Enlightenment thought, without a 
hint of how those hopes were dashed historically by those who were happy to 
oppose collective visions of “progress” and “reason.” 4 Nonetheless, I believe 
that this approach provides a useful normative framework for those attempting 
to think about the structures and behaviors that help large, complex organiza- 
tions respond to changes in their environment, whether these are economic, 
political, or demographic. 

Assessment, in other words, needs to be seen as a form of collaborative 
inquiry, undertaken by those most heavily involved with instruction, and con- 
ducted in such a way that its findings give further direction to those who 
would like to practice double-loop learning to further refine and adapt existing 
disciplinary curricula and teaching practices to the particular needs of their 
students. Even while assessment attempts to analyze and enhance student 
learning at an aggregate, program level, it also indirectly gauges this learning 
as the result of more or less effective interactions between the various stake- 
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holders at an institution: e.g., how well are students able to incorporate the 
writing from their first-year composition courses into their subsequent classes? 
How well can they use the library in their research projects? What percentage of 
a program’s graduates applies successfully to graduate programs? And so forth. 

What I would like to do in the remainder of this essay is to use the devel- 
opment of this CTE project to help illuminate the uses of assessment in a 
number of contexts, contexts that will help to provide a better sense of its pur- 
poses, benefits, and limitations. The first “use” of assessment I will examine 
will be its perennially vexed relation with its institutional shadow, accountabili- 
ty, for which it is continually mistaken and from which it must always distin- 
guish itself. Then I would like use some additional episodes from my own his- 
tory to illustrate how faculty-driven assessment has to work both within and 
between departments in the complex organizational structures of the modern 
university, for the sake of better instruction at the departmental level and bet- 
ter communication among departments at the university level. Both aspects of 
assessment are necessary to rhetorically coordinate otherwise scattered 
enhancements of instruction and to promote better learning and more pro- 
ductive interactions at every level of the organization. My final section uses 
my own background in literary studies to offer a rhetorically based and inter- 
disciplinary perspective on assessment, an eclectic area of scholarship that 
helps disciplines do their own double-loop learning, by providing members 
with a view of its effects on students in their first encounters with its charac- 
teristic forms of knowledge. 

Finally, this essay would betray its commitments to both double-loop 
learning and humanities pedagogy if it did not introduce some kind of discor- 
dant note into its discussion of the faculty role regarding assessment. To para- 
phrase Marx, faculty may be able to make assessment, but they will not make 
it just as they please, under self-selected circumstances, but under circum- 
stances that are to some extent given and transmitted from the past. It is worth 
noting that our faculty senate has been able to push our goals of self-improve- 
ment and enhanced academic reputation, but only within the context of long- 
standing stratifications in Texas state politics and higher education policy. 
Likewise, we have been able to help our institution define its notions of aca- 
demic “quality,” and energetically pursue those goals, but only within a set of 
academic and economic discourses that by no means automatically favor an 
institution like ours. Finally, even as we deploy terms like “assessment,” 
“improvement,” “responsiveness,” or “innovation,” we on the senate are thor- 
oughly aware of the historical suspicion that they provoke among fellow facul- 
ty members, because of these terms’ historical usefulness for the advocates of 
accountability and corporatist management of the university (Birnbaum, 
Management Fads). Given the history of the accountability movement, we can- 
not define these terms however we please. Nonetheless, it seems to me that our 
best hopes as faculty lie with the possibility of using all the rhetorical and con- 
ceptual resources at our disposal to rhetorically rcdescribc and redefine these 
terms in such a way that they can serve our purposes, and help us represent 
the university more effectively to the multiple publics it serves. 5 
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Doing What We’re Supposed to Do, But Only Because We Want to 

One of the most memorable exchanges I had during the course of our 
GTE discussions was at a public forum in which a faculty member asked me 
whether we were doing this project because “we’d been ordered to by the 
provost.” I was taken aback by this question, because though we had used 
data like our six-year graduation rates to argue for the need for a GTE, it had 
never occurred to me that we could be viewed as doing this at someone else’s 
behest, or, as this faculty member explained, because the administration just 
wanted us “to solve this problem for them.” I had always taken for granted the 
idea that faculty would want their institution to be improved, regardless of 
whether they were willing to participate or not. What I had not anticipated 
was a debate over the definition of “improvement,” or who got to define it for 
the university. Nonetheless, this question opened up the whole issue of the 
motives and agendas underlying not just our efforts to improve instruction, but 
any efforts toward improvement in an institution like our own. On whose 
behalf were we doing these things? Who would benefit? And why should oth- 
ers join us? It became imperative that we be able to answer these kinds of 
questions for skeptical faculty members, if we wished to persuade anyone to 
participate in this project. 

In my own case, I learned early on that it was crucial for my credibility not 
to be seen as an advocate for “accountability,” which is omnipresent and much 
loathed in my state’s K-12 system, and which is rapidly expanding throughout 
the Texas higher education system. 6 At the same time, even while assessment 
and accountability measures moved ahead with or without faculty participa- 
tion, there remains very little agreement in- or outside my university about the 
value of either assessment or accountability. Not only do accountability advo- 
cates in the legislature and business community routinely ignore our representa- 
tives, but our own administrators, faculty, and assessment experts rarely agree 
amongst themselves about the nature of either assessment or accountability. 

More tellingly, our discussions on campus, just like discussions I have seen 
in the secondary literature of assessment, featured questions that were contin- 
ually raised but rarely answered to anyone’s satisfaction: assessment experts 
asked, “Why isn’t assessment better integrated into academic culture?”; 
administrators wondered, “Why don’t we use the data we collect?”; and facul- 
ty demanded, “Why should we even be doing this?” (Ewell, “Perpetual 
Movement” 3-4). These seemed to be archetypal roles we were each fated to 
play into eternity. Each group seemed to be entering into the discussion at a 
different phase, with a different level of acceptance and understanding of what 
assessment and accountability might mean in a higher education context. 

The deadlocked nature of this debate can be traced pretty easily back to 
the understandable ambivalence of most faculty members toward assessment. 
They are, after all, the people in the university who are most intimately involved 
with instruction, apart from the students, but who for the most part have never 
committed to assessment as part of their scholarly responsibilities, largely 
because its benefits seem to lie elsewhere. This ambivalence resembles the 
ambivalence I detected whenever I discussed conditions of teaching and 
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learning across campus, because they both involved the discomfort faculty 
manifested when asked to open up their teaching to public scrutiny This was 
true even when the public consisted of other faculty I quickly realized that 
sharing such classroom information always involves a significant loss of con- 
trol, especially when faculty do not know how the information could be used 
or who else might see it. Such self-exposure requires a good deal of accumu- 
lated trust before faculty will feel comfortable enough to share information 
about what goes on in their classrooms. 

Thus, one of the causes for the well-known faculty ambivalence toward 
assessment is not simply its origins in what one assessment expert calls “exter- 
nal bodies” but also because the results of such assessments potentially serve, 
as he notes, “two very different (and at times incompatible) agendas” (Ewell, 
“Perpetual Movement” 2). These consist of an internal agenda of what I have 
been calling institutional self-improvement, spurred on and sustained by inter- 
ested faculty and administrators as stakeholders, and an external agenda of 
“accountability,” fostered by outside forces like state agencies, representatives 
of the business community, accreditation agencies, administrators, and so 
forth." In our case, we on the CTE steering committee were asked so many 
times whose agenda we were pursuing, and how we could guarantee that such 
data would not be hijacked and misused, that we began to stress that establish- 
ing a GTE would help to prevent such a scenario of abused information. Instead 
of serving as an unrcflective instrument of such information-gathering, we 
would provide faculty a professional forum to develop our instructional priori- 
ties, communicate these priorities to the administration, and ensure that the 
gathering of information was directed by representative faculty groups rather 
than administrators. 

We learned other things, too, that helped shape the features of the CTE 
proposal we are currently implementing. For example, we decided very early 
on that this center should serve the purposes of formative evaluation of teach- 
ing (assessment) rather than summative evaluation (accountability). It would 
help provide electronic and other resources for both individual instructors 
(including teaching assistants, and both tenure- and non-tenure-track faculty) 
and larger groups on a voluntary basis, all on the model of “professional 
development” (O’Day 34-36). It would also house and support our existing 
university-wide teaching prize committees, and administer a modest summer 
course-development fellowship for interested faculty. We also hoped to build 
links with the university’s own division of institutional research (IR), so that in- 
house research regarding university policy like admissions standards, retention 
and graduation rates, and student engagement could be conducted collabora- 
tively by interested faculty with members of IR, along with grant develop- 
ment. Finally, the center would also house the university’s now-scattered teach- 
ing assistant (TA) training programs, and assist departments with their training 
and evaluation of graduate students. What all of these activities shared was a 
desire to improve the interactions taking place within departments regarding 
instruction (in areas like TA instruction, for example), but also assisting depart- 
ments as they attempt to enlist help from other parts of the university. In this 
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respect, the CTE would help facilitate communication about instruction 
between different parts of the university, and thereby make the interactions 
themselves more informed and productive. 

Yet the role of this center, in keeping with its collegial and collaborative 
origins, would remain consultative in nature, and would be directed by a facul- 
ty board rather than a full-time administrator, at least in the initial stages of 
implementation (Borden, “Information Support’' 177). In all these respects, 
the establishment of this kind of hub for research and discussion involving 
teaching and learning should translate into better dissemination of our find- 
ings, more general enhancements of teaching, and hopefully an enhanced aca- 
demic reputation, which is, after all, one of the chief motivations for faculty 
involvement. 

Our hope was that the research orientation and teaching experience of the 
faculty involved could introduce better information, more reflection, and a 
broader disciplinary perspective into the university’s internal deliberations con- 
cerning academic policy, and could help us to continue to refine our understand- 
ing of our students and their needs. In this way, the CTE could model organiza- 
tional learning for both our administration and individual departments.® 

However, in order to convince the faculty who remained suspicious of our 
motives and unsure of the value of what we proposed, we engaged in a two- 
year long process of collective deliberation designed to gather input from as 
many stakeholders as possible throughout the university, including two faculty 
forums designed to collect suggestions and preview our proposal, before we 
submitted a fully worked out proposal to the provost in May 2010. Only by 
remaining fully transparent in our deliberations, communicating our inten- 
tions to the university community, and publicizing the results of our efforts, 
could we demonstrate our commitment to serving both the university and the 
faculty as a whole. And indeed, this commitment to communicate our efforts 
and to remain a collegial and consultative rather than an administrative unit is 
what distinguishes our efforts from the accountability measures that are con- 
tinually being devised by the state. 

Doing What We’re Supposed to Do, Because We Have To 

While we on the CTE steering committee have been focusing on the 
improvement of instruction at UH, the state legislature had in the meantime 
mandated its own accountability measures. Texas House Bill 2504 (HB 2504), 
which was passed in 2009, requires that public universities make accessible on 
the Internet the cost of attendance, course syllabi, and student course evalua- 
tions for every course taught on- or off-campus, along with faculty salary 
information.® The difference in priorities is instructive, since the purpose of 
HB 2504 does not appear to be enhancing the quality of instruction, but 
rather to publicize the relative cost of higher education from institution to 
institution, and to promote the notion of the student evaluation as a key indi- 
cator of the quality of instruction. 10 The effect of this legislation, which 
demonstrates the continued alliance between accountability advocates and the 
conservative campaign against higher education in our state, has only strength- 
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ened the faculty’s conflation of accountability with assessment. More damag- 
ingly, it further eroded the trust necessary for accurate information about 
instruction to be freely collected, shared, or communicated, either internally 
or externally. 

What the unhappy example of K-12 education under No Child Left 
Behind (NCLB) has shown, however, is that a non-response to such accounta- 
bility measures is the worst possible choice. 1 1 Those who work at the non-flag- 
ship schools must offer some kind of response to such demands, because such 
demands are not going to go away. Nor can we expect to prevail with a sand- 
bagging, essentially bureaucratic response of outward compliance, since any of 
these non-responses will simply invite additional and more invasive oversight. 
More importantly, the failure of other institutions to respond publicly to these 
demands leaves vulnerable schools like ours in a position wherein we have nei- 
ther improved our academic standing, adapted to the changes in our circum- 
stances, nor affected the views of those making the demands. So the non- 
response response only defers our confrontation with the external forces plac- 
ing these demands upon us, while leaving us with fewer and fewer resources to 
defend ourselves. 

I think that academics at other institutions might consider what is at stake 
in the political, institutional, and rhetorical situation we now find ourselves in 
at UH, where we have been working to improve our institution, and lift our- 
selves out of strict dependence on the state’s formula funding, even while the 
state imposes new forms of accountability almost every year. For one thing, it 
shows that other schools’ sense of independence may be deceptive, as more 
and more states experience fiscal crises in the wake of their local recessions 
and real estate “readjustments.” Even the long-time flagship institutions in 
these states, given an atmosphere of ubiquitous financial crisis created by 
decades of anti-tax policies, can push once-recognized departments into the 
crises we have routinely experienced for over thirty years. This is perhaps the 
main reason why most public universities have been seeking out corporate or 
non-governmental sources of funding since the 1970s, when federal and state 
governments began decreasing their contributions to higher education (Burke, 
“Many Faces” 5-9). 12 

So where does this double bind of lessened support and increased over- 
sight leave contemporary universities, whatever their stature and reputation? 
First, it means that faculty should regard the increasing politicization of higher 
education as a reflection of the public’s genuine interest in the higher educa- 
tion institutions it (under) supports, while also remembering that this concern 
can become distorted if we define the “public” too narrowly, or follow the 
cues of a particular portion of the “public” too exclusively. 13 Second, it means 
that institutions of higher education must learn how to acknowledge their 
responsibilities to the broadest and most representative portions of the public, 
and communicate what we do as transparently as possible to all these groups. 
Third, it means accepting the notion that the politicization of higher educa- 
tion, like the politicization of K-12 education, will be around for a long time 
to come, and will continue to affect our ability to communicate with our mul- 
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tiple publics. In my view, it is essential that we devise new forms of advocacy 
for higher education, to address our multiple publics better, and to garner 
stronger, more widespread, more robust support from the publics we serve. 

In order to become better advocates for higher education, however, we 
need the deep knowledge of our students and the responsiveness to their 
needs made possible by the robust, faculty-centered forms of assessment I 
have been describing. Publicly demonstrating this kind of knowledge and 
responsiveness helps cut down the calls for more accountability and more 
compliance, by preventing the accountability movement from filling up the 
discursive space with uncontested images of do-nothing, disengaged faculty. In 
our case at UH, at an earlier point when the state was considering still more 
intrusive forms of accountability, the senate was able to cite the QEP and 
other initiatives in undergraduate research to demonstrate that faculty were 
heavily involved in initiatives to improve student learning. 

The contrast between our own initiatives to improve instruction and the 
fallout from HB 2504 helps further clarify the differences between assessment 
and accountability, because one gathers its information for faculty to improve 
instruction, while the other is on the whole indifferent to the actual quality of 
learning throughout the university. If accountability serves any purpose at all, 
it exists as a bureaucratic mechanism to alert those outside the university 
whether its resources are being used efficiently and effectively (Burke, “Many 
Faces” 2; 10-13). Whether that degree of efficiency or effectiveness actually 
serves the students, however, is a question that accountability mechanisms can 
rarely answer. For example, students, in spite of their sensitivity to tuition and 
fee increases, are generally less happy to find their access to faculty and 
required courses getting rationed in the name of improved “efficiencies” in 
course scheduling. Thus, the free-market metaphors of many accountability 
advocates, which stress a consumer-model for education, ignore the actual 
consumer behavior of students selecting their colleges on the basis of prestige 
rather than cost (Zemsky). Likewise, in its drive for “efficiencies,” accountabili- 
ty often neglects the needs of non-traditional students who may require addi- 
tional time to complete their degrees. In sum, there is no reason for universi- 
ties to take accountability advocates at their word when they claim that they 
fully or adequately represent the views or needs of the publics that actually 
sustain the university. 

Inside or Outside? 

Given the complexity of modern research universities, I do not think it is 
helpful to make absolute distinctions between “internal” and “external” agen- 
das, or to speak of a “choice” between assessment and accountability. A more 
workable model of the university’s relations with its environment is offered by 
former City University of New York chancellor and higher education researcher 
Joseph Burke, who has identified at least three distinct yet divergent sets of val- 
ues, concerns, and interests that help to govern contemporary higher educa- 
tion, which he designates the “academic, civic, and commercial” cultures, 
respectively (“Many Faces” 9). Burke goes on to argue that these three “legs of 
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the accountability triangle” represent the three cultures that help sustain 
American universities, which include not just the academic and disciplinary 
concerns of the departments, but also two distinct sets of external forces that 
continually fall into conflict with one another, namely civic and business inter- 
ests (21-24). We can recognize these kinds of conflicts in the contradictory 
demands placed upon Texas universities, cither to pay our own way with out- 
side research, or to teach ever increasing numbers of students, or both. 

Burke admits that the persistent tensions between the business and civic 
cultures, and their struggles with one another and with the academic culture, 
make it impossible for universities to opt simply to accept or reject cues com- 
ing from outside (9-10). Clearly, a more politically effective strategy is necessary. 
Yet this mingling of inside and outside forces seems only to have increased 
since Burke’s essay was published six years ago, while declining public support 
for higher education, increased corporatization, and growing demands for 
accountability have only accelerated. 

Nonetheless, no matter who gets assigned the status of “insiders” or “out- 
siders,” Burke’s comprehensive list of the tensions between internal autonomy 
and external accountability in the academy still seems pertinent to the discus- 
sion of improvement and its means. Burke’s list of tensions between “internal 
interest” and “external concern” includes the following pairs: 

• Institutional improvement versus external accountability 

• Peer review versus external regulation 

• Inputs and processes versus outputs and outcomes 

• Reputation versus responsiveness 

• Consultation versus evaluation 

• Prestige versus performance 

• Trust versus evidence 

• Qualitative versus quantitative evidence. (10) 

Though I believe that Burke’s hope of reconciling the interests of accountabil- 
ity and institutional autonomy is unrealistic, I agree with his point that this set 
of tensions would confront any university attempting to improve itself. Every 
one of these pairs represents choices that admit no definitive answer or pre- 
scription: who can say when trust necessarily outweighs performance, or when 
reputation trumps responsiveness? 

From the perspective of assessment, the most noteworthy aspect of 
Burke’s list of tensions is the fact that neither the qualitative evidence encour- 
aged by assessment nor the quantitative evidence favored by accountability are 
sufficient to satisfy the informational needs of the contemporary university. 
Both constitute evidence of student learning, and both are necessary for the 
purposes of improvement and publicity. They each provide distinct though not 
self-sufficient views of instruction, and each will be useful to the various inter- 
nal and external audiences for such information. Nonetheless, it seems neces- 
sary to reiterate that large-scale institutional improvement seems far more like- 
ly to come from an internally driven process of assessment and organizational 
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learning than from the essentially bureaucratic practice of compliance with 
outside accountability measures (O’Day 28-9). 

What Burke’s list ultimately suggests, however, is that one of the most 
important roles of organizational learning in higher education lies in guiding 
institutions so that every stakeholder — administrators, faculty, and others — 
learns how to develop an appropriate form of responsiveness that is not simply 
the reflexive following of cues (whether these come from business interests, 
legislators, students, etc.), but represents a more dynamic, thoughtful, self- 
regulated relation to one’s environment (“Many Faces” 23). In the complex, 
changing environments envisioned by Argyris, organizations that lack such 
self-regulated thinking quickly follow the cues of a single stakeholder straight 
into catastrophe. 

In contrast, a self-regulating university with the capacity for learning 
would be capable of selecting, and adapting to, whichever external demands 
might help it thrive under changing circumstances, and would learn how to 
rhetorically counter, parry, or minimize the demands that would jeopardize its 
reputation, its quality of instruction, or responsiveness to other stakeholders. 
This, in any case, is the normative framework set out by Burke. As he asserts, 
“Higher education [is] inevitably accountable to state priorities, academic con- 
cerns, and market forces. [Its colleges and universities] should serve all while 
submitting to none of these imperatives. Being accountable to each of the 
three corners of the Accountability Triangle means balancing the response to 
ensure service without subservience to public priorities, academic concerns, 
and market forces” (23). This tactical and strategic autonomy, practiced within 
the constraints of existing public support and servicing public needs, repre- 
sents what I would regard as the most plausible model of faculty governance 
at this historical juncture. 

This kind of informed balancing is also something faculty should learn, as 
well, if they wish to keep their own leadership accountable to them for the 
direction of their departments, colleges, and universities. More realistic expec- 
tations, for both rank and file faculty and their chairs and administrators, lead 
to more consistent and more useful evaluation of their performance, and the 
presumption that both sides will work harder to respond to those high but not 
impossible expectations. Thus, “responsiveness” in this expansive sense can be 
seen not just as an important principle to uphold for external relations, but as 
part of what makes universities function internally. 

Opening Up the Boxes to See What’s Inside 

When I first began describing the possibility of a GTE to my fellow facul- 
ty members, I was surprised and puzzled to receive comments that this was 
only a device for spying on faculty. Terms like “teaching police,” “teaching 
jail,” or (my favorite) “authoritarian fascist monster” were representative of 
the emotions initially provoked by our proposal, which I naturally regarded as 
quite benign until I saw the defensiveness it inspired. It was only when I began 
learning more about the scholarship of assessment that I was able to recognize 
that these highly negative responses assumed that the CTE’s primary purpose 
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would be administering accountability rather than serving as a faculty-based 
resource for improvement. Once again, the recurrent anxiety I encountered 
taught me how important it was to establish a “safe place” to discuss teaching 
on campus. To sustain such a forum, we would need to build trust about how 
our discussions would be used, and emphasize that our information-gathering 
was strictly for the sake of formative and consultative feedback with faculty 
and departments. 

I also found myself repeatedly having to distinguish between the standard- 
ized K- 1 2 accountability measures so popular throughout Texas public schools 
and the decades-old traditions of higher education assessment, which seemed 
far more pluralistic and receptive to the humanities and performing arts 
(Ewell, “An Emerging Scholarship”). It was during these discussions that I dis- 
covered a whole vein of education policy scholarship about the differing impact 
of accountability measures like NGLB on various types of schools, and the 
kinds of institutional cultures, histories, and characteristics that either encour- 
aged or discouraged organizational learning (Abelmann and Elmore; Elmore, 
“Problem of Capacity”; Fuhrman and Elmore). 

One of the leading exponents of this approach, Harvard education policy 
expert Richard Elmore, has written eloquently about the divergent effects and 
long-term consequences of such external demands for improvement, when 
these are placed upon schools in vastly different situations, in terms of com- 
munity placement, student body characteristics, financial and other resources, 
internal norms and structures, and so forth (Elmore, “Problem of Stakes”). 
Elmore’s work helped me recognize that many of the difficulties we experi- 
enced at UH with accountability measures stemmed from the imposition of a 
single, uniform standard statewide upon very different institutions with corre- 
spondingly different missions, student bodies, and available resources. 

More importantly, Elmore also taught me how to talk with my faculty 
about using our proposed CTE as a place to build up “capacity,” in the form 
not merely of infrastructures, technological support, and so on, but also in 
terms of knowledge and expertise, as produced by continual inquiry to learn 
the best ways of reaching our students. In many respects, since our current 
way of doing things in the classroom had left us with demonstrably unimpres- 
sive results (that fourth-tier USJVWR ranking again), the only hope of change 
lay in learning how to do things differently, and better, than we were currently 
doing. These efforts towards producing large-scale change, I would argue over 
and over again, would be much easier to accomplish with the existence of a 
centralized forum and resource center like the CTE (Elmore, “Problem of 
Capacity” 248-58). 

At the same time, Elmore identified one of the most frustrating aspects of 
this kind of organizational work when he noted that 

it is commonplace to conceptualize learning and improvement in 
school systems as a kind of Chinese box problem, or a problem of 
“nested learning communities.”... Learning that is instrumental 
to improvement occurs at the individual level, at the group level 
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within schools, at the school level, and at the system level; each 
level of learning is required to support the others; learning at 
each level is the reciprocal of learning at all other levels. So 
learning at the individual level — teachers and principals, for 
example — is instrumental to improving learning of students, but 
students provide the feedback that is necessary for adults to learn. 
(“Problem of Capacity” 251) 

Though Elmore is referring here to the organization of American high 
schools, the collegial faculty- and department-centered organization of higher 
education, along with a long tradition of department-centered governance, 
makes it that much more difficult for individual departments to learn and 
adapt to new circumstances; each department holds itself accountable not just 
to the university, but also to a national or even international disciplinary net- 
work or “disciplinary community” that maintains its own standards over and 
against local “campus community” and its demands (Alpert 250-59; Elmore, 
“Problem of Capacity” 251). This complicates any kind of local, horizontal 
communications that might take place around the one topic that these verti- 
cally stacked disciplinary units otherwise share: undergraduate teaching 
(Alpert 267-69). 

The final insight I took from Elmore’s work was that accountability initia- 
tives, for better or worse, force every institution, regardless of its prior history 
or current status, to confront the prospect of improvement, but that some 
institutions were far better equipped than others to organize themselves 
around that goal. Elmore’s research has shown that the key difference lies in a 
“construct” he calls an institution’s “internal accountability,” which he defines 
as an existing and coherent set of internal norms, structures, resources, and 
attitudes that permits the institution to work in concert with itself to accom- 
plish the work of improvement; this construct of internal accountability can 
either motivate or obstruct organizational learning throughout a school 
(“Problem of Capacity” 246). This is because success at a task of this scale 
demands that an institution coordinate its members’ activities, so that each 
member helps to advance and sustain its collective mission within her particu- 
lar domain. The only way to accomplish this kind of coordination, however, is 
by communicating, tacitly or otherwise, the interlocking set of responsibilities, 
expectations, and norms that individuals must conform to if they wish to 
remain in good standing. While these responsibilities and norms could be set 
either high or low, institutions with incoherent internal norms experienced far 
greater struggles with the learning, communication, and adaptation necessary 
to alter their current, unsuccessful practices. 

And yet, unsurprisingly, perhaps, Elmore found that it was far more com- 
mon to find schools that were “atomized,” or incoherent, or where “individual 
preferences account for most of what happens in the organization” (“Problem 
of Capacity” 246). Though an institution’s failure to reach this kind of conver- 
gence might have few consequences under generally stable conditions, this 
lack of convergence can still have catastrophic consequences when vulnerable, 
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fractious, or self-divided institutions are suddenly hit with external demands 
that tax their capacity to respond. 

What Elmore’s research makes clear is that any demand for improvement, 
whether internal or external, can have as negative an effect as an economic 
shock on an organization’s infrastructure, if it is not prepared to put the 
demand into effect. As Elmore explains, 

pushing hard with an external force — such as testing and sanc- 
tions — on an atomized organization . . . does not make it a more 
coherent organization ... in fact, it often makes the organization 
more atomized and dysfunctional because people continue to do 
what they know how to do, and which is exactly what produced 
the performance that got them in trouble in the first place. 

(“Problem of Capacity” 246) 

This seems to me a helpful way to understand the panic that sometimes 
emerged out of our discussions. We were witnessing the recognizable panic 
that overtakes us all whenever we are asked to learn something new; this 
panic is compounded when we feel that we are not being given the time and 
resources to learn a new area properly. It is unsurprising, then, that our calls 
for improvement sometimes provoked the kinds of defensive reasoning iden- 
tified by Argyris and Schon, though this defensiveness was often accompa- 
nied with extraordinarily useful feedback about local problems that were 
otherwise never acknowledged. These panicky encounters with faculty, how- 
ever, taught us to keep talking long enough for forum participants to recog- 
nize the local problems that needed to be addressed, while showing them 
how these local problems reflected more systemic issues happening through- 
out the system. 

As Elmore notes, one result of this dynamic is that schools with low inter- 
nal accountability — or as Argyris would say, well-established “defensive rou- 
tines” (214) — tend to respond to such external demands by “managing] 
around the edges of instructional practice, preserving an essentially atomized 
organization” (“Problems of Capacity” 247). This kind of outwardly compli- 
ant behavior “leaves as much as possible of the existing organization intact” 
(247). In contrast “schools with high internal accountability have norms about 
what good teaching and learning look like,” and are able to accommodate a 
variety of external demands (e.g., testing, etc.) and perform well on them, 
“with a minimum of disruption to their internal processes” (247). In other 
words, schools capable of learning are able to accommodate even the change 
represented by accountability measures. Thus, the first step toward organiza- 
tional improvement in even the lowest performing schools is not some free- 
market style “creative destruction” of existing practices, but better internal 
organization, an internal and rhetorical alignment of norms and expectations 
around shared goals. Aligning norms and expectations in this way, as well as 
building up institutional capacity, helps organizations learn what they need to 
do differently to improve instruction (O’Day 19). 
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At this point, however, we should note how differently a term like “self- 
improvement” resonates at a longtime tier-one school, compared to its effect at 
a “Wannabe U,” which is a school defined more by its aspirations to excellence 
than by its secure possession of those qualities (Tuchman). In that single word, 
“aspirations,” we can find all the pathos and all the frustrations of working at 
Wannabe U. Consequently, far more risk, and far more anxiety about poten- 
tial loss of control and reputation, accompanies the admission of the need for 
self-improvement at Wannabe U, when compared to schools that are already 
known for a high quality of undergraduate education. This is one of the rea- 
sons why building trust and communication among faculty is crucial to getting 
the entire enterprise of improvement started on the right footing. At the same 
time, the fact that so many of the terms we might use to describe improve- 
ment have already been co-opted by corporate management fads and educa- 
tion consultants makes it all the more crucial that faculty help redefine terms 
like “improvement,” “responsiveness,” and “excellence” in ways that make 
sense in their own institutional, local, and regional settings. 

The discomfiting example of Wannabe U should teach everyone , not just 
those working at the innumerable Wannabes scattered all over the country, 
that aspiration is one of the most important yet also destabilizing motivators 
we have for education, whether we are talking about individuals or institu- 
tions. Aspiration is to some extent rooted in one’s dissatisfaction with one’s 
current condition, and it is powerful enough as a motivator to persuade other- 
wise risk-averse people to undertake a self-transformation whose final effects 
cannot be predicted. This is a drama I see playing out all the time in my class- 
rooms, and throughout my university. 

The real tragedy, however, is when this admirable willingness to assume 
risk gets diverted, as it does at Wannabe U, into the false hopes and pseudo- 
transformations of the commodified and bureaucratized version of higher 
education instigated by the professional administrative caste of Wannabe U. 
What Wannabe U reminds us is that a form of public “responsiveness” that 
consists of blindly following the dictates of the marketplace is doomed to satis- 
fy no one, and that no version of institution-wide “improvement” can ever be 
conjured up by administrative fiat, if it is lacking the commitment and partici- 
pation of the faculty who must put it into place. What Wannabe U offers 
instead is a dystopian portrait of the contemporary research university where 
there is no genuine assessment, no interest in genuine self-improvement, no 
organizational learning, only market-driven, administrator-imposed, bureau- 
cratically enforced accountability. And who would want to work there? 

Assessment Begins at Home 

Nonetheless, for all our talk about improvement within the university as a 
whole, the entire enterprise stands or falls on the basis of our interactions with 
students in the classroom and with our colleagues in our departments. As I 
have already noted, however, the vertical, “siloized” disciplinary and research 
orientations of higher education, as well as the incentive structures that 
reward this vertical orientation, reinforce the individualism and atomization 
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that allow faculty to neglect both internal accountability and organizational 
learning. It is in faculty members’ interest to define their departmental respon- 
sibilities solely in terms of their disciplinary identities (e.g., “I cover the 
Victorian period and the nineteenth-century novel.”). This exclusively discipli- 
nary understanding of one’s responsibilities also makes it more difficult for 
faculty to accept any degree of oversight for their teaching, to conceive of the 
curricula in- or outside the department as a product of their collective endeav- 
ors, or to deliberate among themselves about the shape or direction of their 
curricula and programs. 

For this reason, it is often difficult for faculty to conceive of their own pro- 
fessional activities as adapting to and interacting with one another, or to rec- 
ognize how their teaching or curricula could interact with the local environ- 
ment of the university or other campus communities, a fact that has been rec- 
ognized ever since the drive to reform undergraduate education began in the 
1980s (Alpcrt 267-69). In this respect, it seems that a non-departmental, infor- 
mal, collegial and consultative forum like our proposed CTE is the best solu- 
tion to this blind spot in our organizational structures. Such a conception of 
the GTE would be able to advocate for instruction across departments, even 
while individual departments and colleges are focused on the problem of keep- 
ing their units whole, not a negligible task in an era of pervasive budget crises. 

Yet it is also true that every member of what one scholar has termed the 
“peer reviewed” professions has a responsibility not only to research and teach 
within a particular discipline, but also to oversee the curriculum offered by her 
and her unit to students (Hamilton). It is in this necessarily divided view of 
faculty members, as individuals teaching specific classes, but also as a group 
contributing to a collectively produced curriculum, that the role of assessment 
can truly come into play. 

Thus, one of the few activities capable of rhetorically aligning views of 
instruction throughout the university is a regular, principled discussion of cur- 
riculum, teaching, and instruction, along with evidence of its effects, and con- 
tinual efforts towards their improvement. For the purposes of this collegial dis- 
cussion, faculty need to supplement their own individual experiences of teach- 
ing with what Peter Ewell has termed “locally designed and operated evalua- 
tion research,” focused on the effects of their overall program on their stu- 
dents, and “centered on learning outcomes,” for the purposes of improving 
teaching and learning within the department (“Can Assessment Serve 
Accountability” 1 05). In other words, it is what we have been arguing for all 
along: assessment as institutional self-improvement. 

This notion of a faculty- and departmental^ based assessment practice is 
particularly important for ensuring that assessment takes place in a regular 
rhythm set by faculty, collected within departments, and regularly revisited 
during departmental discussions of curricula and their effectiveness. This has 
the potential to move discussions of curricula out of sterile turf battles or ad 
hominem assaults on particular personalities. It also means that faculty and 
departments can begin to align their own norms, values, and expectations 
around improvement in ways that can be linked up to similar initiatives taking 
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place in other parts of the university. But the principle here is that assessments 
need to take place and get processed at the level where many critical decisions 
get made — at the departmental level. But what would this look like? 

In his essay, “Some Plain Talk about Assessment,” Alan Meyers provides 
an extremely useful description of how he has incorporated assessment into 
his everyday decisions about teaching and instruction, by showing that real 
assessment, not the accountability demands we are more familiar with, is 
“simply another name for what we already do: asking ourselves, are the stu- 
dents learning, and, if they are not doing as well as would like, how can we 
help them do better?” (20). Meyers’s point is that we are thoroughly familiar 
with the concept and pedagogical purposes of feedback, since we consistently 
use it in our own teaching to help students improve. So why not elicit better 
feedback to improve our own teaching? 

Meyers urges us, then, to begin obtaining this feedback in order to inte- 
grate it more fully into our work as teachers, especially as faculty charged with 
overseeing instruction in our departments. Meyers, however, knows that many 
literature faculty are unsure of how they might proceed with such informa- 
tion-gathering, or how to put the information to work. The virtue of Meyers’ 
article, however, is to show that this kind of very small-scale, local assessment 
involves familiar activities, activities like formulating questions and analyzing 
grades in the aggregate, but brings them into a new context, the context of 
our own classroom instruction, which can then be scaled up to larger groups 
of students and faculty and larger-scale tasks of assessment. Though Meyers 
does not use the term, he is clearly trying to get his faculty readership to rec- 
ognize this as the challenge of transferring knowledge gained in one, familiar 
area, to a new and unfamiliar domain. 

Consequently, Meyers provides a very useful set of strategies for assessing 
our impact as faculty members at four distinct levels: in everyday classroom 
interactions, and at the course, departmental, and program levels. At the heart 
of his method is a regular, periodic process of (self- or group-) questioning 
designed to garner and then respond productively to the results concerning 
the instruction we are involved with. According to Meyers, the inquiry process 
begins with a simple set of questions: 

Asking, What do I (we) want students to do? 

Then asking, Are they doing it well? 

If they are not, change something in the instruction. 

Then, see if the change makes a difference (This requires evi- 
dence, usually apart from an instructor’s course grades.). (21) 

Here, I would make two observations. Meyers’ questions oblige us to learn 
more about the effects of our teaching, and demands that we do this by looking 
beyond our own individual acts of grading, and from a consciously different 
perspective. And, of course, the potential evidence for such effects could be 
either quantitative or qualitative, depending on what we were looking for. It 
could be as simple as asking whether majors who took sophomore-level cours- 
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es were later prepared for the courses in the major, or whether graduate stu- 
dents’ coursework was leading to satisfactory performance in their compre- 
hensive exams. And this kind of question leads us to ask about student per- 
formance in the aggregate, as a group moving through a particular course, or 
tier of courses, to see if larger patterns emerge that the instructor or depart- 
ment should attend to. 

As Meyers notes, this process creates a feedback loop that never ends, “as 
we strive for continual improvement of different outcomes and courses” (21). 
His heuristic is simple enough to use on one’s own courses in an informal way, 
but still powerful enough to help focus faculty committees undertaking depart- 
mental level reviews about the effectiveness of particular tiers of courses (e.g., 
Is the methods course working? Do we need a capstone? etc.). 

The key is recognizing that the feedback loop gives important clues about 
the next step to take for those who wish to develop more effective forms of 
instruction. Argyris’ notion of organizational learning, however, suggests that 
the greater the potential impact of the change, the greater the need for an 
inclusive discussion of the proposed change and its potential effects. 

Moreover, if faculty are going to have any input into the collection of per- 
formance data or the implementation of results, it will have to be concentrat- 
ed on the level that one recent article identifies as the “core of faculty gover- 
nance”: the academic unit, meaning the department (Richardson and 
Smalling 70-71). And yet how many opportunities do faculty have of learning 
what happens in other faculty members’ classrooms? Faculty responsibilities 
for aspects of instruction like curricular development, evaluation of teaching, 
and undergraduate education policy mean that without such information, they 
will find themselves debating many issues of instruction without first-hand 
knowledge of conditions. So inserting some degree of unobtrusive data collec- 
tion within departments, with the explicit intention for it to be discussed and 
acted upon within the department by faculty-run committees, makes it far 
more likely that it will be used than data collected by outsiders. And, of 
course, this data can also be shared with the rest of the university and the gen- 
eral public, when departments need to explain the value of their work to oth- 
ers in- and outside the university. 

Rhetoric and Interdisciplinary Learning? 

One of the biggest obstacles to assessment being truly integrated into our 
discipline’s instruction and curriculum is the still-common assumption that 
assessment represents an autonomous specialty of its own, complete with its 
own quantitative, social science methodology. Interestingly, this perception is 
less common within specialist discussions of the scholarship of assessment 
than in statehouses and conservative advocacy groups, though there remain 
segments of the university where these methodological battles are still being 
fought. 15 Yet this seductive notion of an autonomous assessment allows both 
faculty and administrators to assume that the best kind of assessment takes 
place apart from instruction, so that all instructional activity remains unin- 
formed by either the assessment process or its results. This self-isolating notion 
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of assessment, like the brown envelopes I still receive with the previous semes- 
ter’s evaluations, becomes all too easy for faculty and departments to ignore. 

As I have mentioned earlier, the major problem with this division of labor 
between assessment and instruction is not only that stealth assessment is con- 
ducted at a distance from those most affected by it, but also because it denies 
faculty the opportunity to help learn from the process of inquiry. This division 
of labor allows assessment to be run solely by administrators and staff rather 
than by the faculty, an arrangement better suited to the hierarchical practices 
of accountability than the collegial practices of assessment. 

My suggestion is that assessment, chiefly in the form of the local and col- 
laborative exercises described by Meyers, should become more like grading, 
operating in effect as a scholarly protocol of information gathering maintained 
by the individual instructor, determined by disciplinary context, and unre- 
stricted to a single group of specialists on campus (though departments would 
certainly need assessment professionals to continue their own institution-wide 
activities, even while they collaborated with faculty on local assessment proj- 
ects). This multidisciplinary, collaborative view of assessment, however, also 
entails that the accountability movement’s privileging of quantitative over 
qualitative understandings cannot hold for work done in the humanities and 
the arts. 

Nonetheless, to keep this version of assessment from becoming too mushy 
and self-referential, I would also endorse assessment experts Marcia Mcntkowski 
and Georgine Loacker’s basic features of a “collaborative scholarship of 
assessment,” namely that 

• The activity (i.e., the collaborative scholarship) requires a 
high level of discipline-related expertise. 

• The activity breaks new ground, is innovative. 

• The activity can be replicated or elaborated. 

• The work and its results can be documented. 

• The work and its results can be peer-reviewed. 

• The work has significance or impact. (83) 

The challenge, of course, is that the forums in which such collaborative schol- 
arship of assessment would be practiced, evaluated, and disseminated would 
also be multidisciplinary, though focused on creating common, or at least over- 
lapping, “conceptual frameworks” concerning assessment and student learning 
(89-91). And, of course, the natural site for the generation of such common 
conceptual frameworks, in the light of local problems with instruction, would 
be a collaborative, multidisciplinary forum like the CTE. 

Finally, a word about “rhetoric,” a practice that every member of the 
GTE steering committee relied upon throughout a year’s worth of discus- 
sions. Throughout this essay I have been implicitly aligning my collaborative, 
faculty-driven notion of assessment with the version of interdisciplinarity initi- 
ated by the “rhetoric of inquiry” movement, which studies how scholars com- 
municate among themselves and to people outside their disciplines (Nelson et 
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al. 3). 16 This approach dictates a careful engagement with the evidence, con- 
ventions, and forms of argumentation conducted within a particular disci- 
pline, while still judging on a case-by-case basis each discipline’s competing 
claims to unity, completeness, self-sufficiency, or dominion over a particular 
field of human activity or thought. This seems to me to describe the kinds of 
interactions we necessarily had while trying to persuade members of our 
home and other disciplines about the necessity for better, more coordinated 
forms of instruction. It represents yet another instance of both interaction and 
organizational learning, wherein the disciplines in a particular setting aid one 
another in a commonly conceived enterprise. 

As Bruce Robbins pointed out some time ago, it is precisely the incomplete- 
ness of every disciplinarily defined field that allows other disciplines to engage 
productively in a rhetorically mediated interdisciplinary inquiry, and that 
allows the results in turn to be submitted to a public that includes of course all 
the other disciplines (Robbins 1 16; Mailloux 25). As Robbins says, “What pro- 
duces an incongruity within any field, the presence of the Other in it that 
keeps it from being entirely and complacently itself, is precisely ‘the public,’ 
that is, the discipline’s need to legitimate its existence vis-a-vis other disciplines 
and society at large” (1 16). Robbins’ notion of an “incongruity” or “presence 
of the Other” is a useful reminder that the desire to legitimate one’s existence 
is not an admission of disciplinary weakness (the usual literary-critical anxiety 
about not being a “real discipline”), but an acknowledgment of the constitu- 
tive incompleteness of one’s own field of knowledge that allows it to be 
reviewed and supplemented by all the other disciplines. 

These questions of rhetorical mediation and disciplinary incompleteness, 
in turn, are also relevant to the status of assessment as an academic movement 
and as an emerging field of scholarship. When distinguished from the more 
bureaucratic forms of accountability, assessment at the university level repre- 
sents a historically novel form of inquiry into disciplinary learning that engages 
critically and productively with other disciplines’ preferred self-images of sci- 
entific objectivity or humanist ineffability. Assessment could provide informa- 
tion that would challenge and complicate these institutional self-images and 
trouble the autonomy and self-sufficiency of each field. 

At the same time, however, assessment itself represents an eclectic, still 
provisional bricolage of elements from fields like educational psychology, scien- 
tific management, and other scholarly traditions that were assembled specifi- 
cally for the purposes of analyzing instruction done in specific fields and insti- 
tutional contexts (Ewell, “An Emerging Scholarship” 5-7; Gray 50-51). In this 
respect, assessment constitutes a form of local knowledge assembled by its 
practitioners, as well as a built-in challenge to the insular local knowledges of 
the disciplines whose teaching practices it would evaluate. 

To return to the puzzle over the unintegrated status of assessment in the 
academy, the “perpetual movement” Ewell finds between internal and exter- 
nal forces should not be seen as a sign of its weakness or lack of scholarly 
integrity, but signifies one of its most crucial functions in the contemporary 
research university: its rhetorical and mediating function for rendering discipli- 
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nary knowledge and disciplinary instruction accessible to those who may never 
experience such teaching first-hand. This is how assessment, when conceived 
in such an interdisciplinary, scholarly, and rhetorical fashion, helps to produce 
what Robbins terms an 

opening of what appears to be private in disciplines to public 
scrutiny and public accountability. This task [lie continues] could 
be described as “public-making”: making public or visible, open- 
ing to a variety of perspectives and judgments, but also the inter- 
disciplinary fashioning of new publics, new instances of judg- 
ment, new collective viewpoints. (116) 

It is the openness and expansiveness of these “new publics,” however, that 
takes the chill out of Robbins’ un-ironic uses of terms like “public scrutiny 
and public accountability,” and aligns Robbins’ account more closely with the 
guarded democratic optimism of writers like Michael Warner or Charles 
Taylor. 17 At the same time, Robbins’ “public scrutiny” could not be further 
from the free-market populism of accountability advocates like former 
Secretary of Education Margaret Spellings, former University of Texas regent 
Charles Miller, or the Texas Public Policy Foundation. 18 In the wake of the 
increasing politicization of higher education, literature faculty need to address 
and respond to the public’s concerns about the responsiveness of their univer- 
sities through mechanisms capable of communicating, explaining, and per- 
haps even justifying disciplinary knowledge to those outside academia. 

I would like to thank Donna Heiland and Laura Rosenthal for getting me to start thinking 
about this topic and inviting me to participate in their MLA panel on assessment. I would 
also like to acknowledge the members of my UH writing group (Maria Gonzalez, Margot 
Backus, Hosam Aboul-Ela, and Karen Fang) for their responses to an early draft of this 
essay. My colleague Jim febro ski gave me some invaluable bibliographic and editorial advice 
along the way. Finally, the members of the UH CTE Steering Committee and CTE Board 
of Directors (Dan Wells, Jim Garson, Cathy Horn, Monica McHenry, and Anadeli 
Bencomo) have been a very helpful audience as I developed the ideas for this essay over the 
past two years. 


NOTES 

1 SACS, like many other regional accreditation agencies, requires its report- 
ing institutions to develop a Quality Enhancement Plan addressing, in the 
words of the Commission on Colleges Resource Manual , a “well-defined 
topic or issue(s) directly related to student learning” (21). In our case, the 
topic chosen was undergraduate research. 

~ The creation and implementation of this research-intensive course has 
been documented in an article I co-wrote with UH Librarian Ms. Julie 
Grob, ‘“Little Living Libraries’: Collaborations between Faculty and 
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Special Collections Librarians in the Inquiry-Driven Classroom,” forth- 
coming in portal: Libraries and, the Academy. 

^ For example, in the Brookings Institution report, “The State of Metro- 
politan America,” the Houston-Sugarland-Baytown region, which is the 
6th fastest growing region out of the 1 00 metropolitan areas studied, 
scored 92nd out of 1 00 in high school attainment, and 54th in completed 
BA degrees. 

' I would add that there seems to be an important affiliation between 
Argyris’ notions of “antilearning,” “defensive reasoning,” and “skilled 
ignorance” (2 1 3) and the post-Enlightenment notions of cynical reason 
and rationalization I explore in my Making of Modern Cynicism. 

^ For rhetoric as a form of tactical rcdescription, see my Cynicism. For a 

view of rhetoric as an “art of the weak,” see Michel dc Certeau, Practice of 
Everyday Life (37-38). 

® For the historical importance of the Texas model in the formulation role 
of the national No Child Left Behind (NCLB) legislation, see the essays in 
Rebell and Wolff. 

^ This notion of the potential conflicts between “accountability” and 

“improvement” has become a staple of many of these discussions of the 
competing interpretations and implementations of assessments. See, for 
example, Banta, “Can Assessment for Accountability Complement 
Assessment for Improvement?” as well as the essays in her edited collec- 
tion, Building a Scholarship of Assessment. 

® Those curious about the current implementation and organization of our 
CTE may find it on our website. 

^ The full text of HB 2504 can be found on the Texas Legislature website. 

1 0 The Texas Public Policy Foundation (TPPF), a conservative think tank, 
was the primary supporter for this legislation. See Young, for example. For 
some of the background, see the TPPF’s website. For a summary of their 
goals for higher education, see Lutz. For a more extensive discussion of 
their longer-term goals, see Vcdder and Denhart. For a cogent response to 
legislative interference in governance issues, see Simmons and Floyd. 

1 1 Though NCLB was passed with the understanding that “increased 
resources for education essentials would be provided in return for increased 
accountability,” these increases have not covered the costs of testing and 
administration, let alone the improvements many educators anticipated; it 
was the hope of these additional resources that silenced or divided educa- 
tion advocates otherwise suspicious of NCLB. See Rebcll and Wolf xv. 

1 ~ It is worth noting, however, that an atmosphere of continual budgetary 
crisis is not an unforeseen event in conservative governance, but represents 
a deliberate and recognized conservative political strategy to shrink public 
institutions as far as politically possible. 

1 ^ In respect to the conservative assault on higher education, I endorse much 
of the critique offered by Marc Bousquet. However, it is unclear to me 
how Bousquet expects his critique of higher education to reach and per- 
suade a larger, less committed segment of the public to reverse the trend 
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he deplores. The only hope that I can see of movement on this question is 
identifying a segment of the public not committed to the right-wing assault, 
and persuading this segment to support politicians committed to greater 
access to high quality higher education. 

^ See Elmore’s definition of his comparative approach: “Institutional 

response theory . . . holds that organizations of all types . . . vary along a 
number of dimensions that affect their responses to external forces in 
their environment . . .” (“Problem of Capacity” 246). Though the discipli- 
nary and governance structures of higher education obviously make for 
significant differences between K-12 schools and universities as organiza- 
tions, the fallout of the accountability movement has been studied much 
more extensively for K-12 than in higher education, and so I have drawn 
on that scholarship to reflect upon the differential impact of accountabili- 
ty on the lesser research universities. 

i r ' 

13 See Ewell, “An Emerging Scholarship” 17-18; Mcntkowski and Loacker 
90. For the continued struggles of writing instruction with quantitative 
assessment professionals, see Huot. 

1 ® From the same Introduction, see also this statement: “Every field is 

defined by its own special devices and patterns of rhetoric — by existence 
theorems, arguments from invisible hands, and appeals to textual proba- 
bilities or archives — themselves textures of rhetoric” (Nelson et al. 5). 

1 ^ See Warner; Taylor. 

1 O ' 

° For Spellings, see Ewell “Assessment and Accountability” 11-13; for Miller 
as public advocate of accountability, see Reyes and Rincon 49-50; for the 
TPPF, see note 10. 
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FROM SKEPTICISM TO MEASURED ENTHUSIASM: 
THE STORY OF TWO LITERARY SCHOLARS’ 
INTRODUCTION TO ASSESSMENT IN THE MAJOR 

KIRSTEN T. SAXTON AND AJUAN MARIA MANCE 


THIS ESSAY CHARTS OUR SHIFTING attitudes as English professors, 
from trepidation to measured enthusiasm, for the value of assessment in our 
field of literary study. Although the English department at Mills College had 
participated in an assessment of the College’s general education composition 
course, the 2009 assessment was our first assessment of a literature course. 
How, we asked, could one quantify the sublime and alchemical nature of the 
teaching of literature? The essay narrates an assessment of our department’s 
entry-level course, “Introduction to Literary Studies,” a course designed for 
first and second year students in the English major, and charts our shifting 
attitudes toward the usefulness and generative potentiality of assessment of lit- 
erature courses. We are each associate professors of English at Mills, a small 
liberal arts women’s college in Oakland, California. In the first section of our 
essay, Kirsten Saxton explains the department’s concern about applying quan- 
titative measurements to a literature course and the steps we took to insure 
that our values informed the assessment process at every stage of its develop- 
ment, resulting in our discovery that assessment can be consistent with our 
shared department values. In the second section, Ajuan Mance speculates 
about what might build on our firm foundation, exploring a vision of the 
potential for assessment to be a transformative tool in the discipline of literary 
studies, and considering the ways in which assessment might be used as a gen- 
erative method by which to make diverse texts and, more importantly, diverse 
theoretical frameworks and approaches meaningfully inherent to the study of 
literatures. 

Part 1: Beginnings: Skepticism and The “Assessment Lady” Bogie 
Kirsten T. Saxton 

I share an anxiety with a creative writing professor at Mills that I think is 
quite telling: in response to our department chair’s thoughtful and excited 
request that we submit a proposal to the Modern Language Association 
(MLA) for a panel on our assessment work, both of us demurred with a 
bemused worry about being marked as “assessment ladies.” Before turning to 
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my topic at hand — my department’s assessment of its (then-named) “Introd- 
uction to Literature” course and how this work fits in with our larger depart- 
mental assessment project — it is worth unpacking this term and its partly 
humorous and partly serious underpinnings. First, to be an “assessment lady” 
is to be marked within the institution, and in the outside world, as one of 
“them” (not just as an administrator, bad enough, but an “assessment per- 
son”). While our official “assessment person” is a universally respected and 
well-liked administrator (no mean feat indeed!), no faculty member I know 
wants to be labeled with the dread tag. To be seen as an “assessment person” 
both implicates one as somehow complicit in what even those of us who are 
generally “for” the project experience as the administrative time-suck burden 
of assessment, and labels one as less serious within one’s own scholarly or cre- 
ative endeavor — the standard curse of administrative association writ even 
larger by the particular dystopic associations of assessment. And the “lady” 
term, while desultorily thrown out, is telling: assessment work has a tinge of 
the domestic, the picking up the socks of the institution, an anti-intellectual 
service component that, despite our logical and philosophical refusal of it, has 
real emotional and political weight in the arenas of higher education. 

In this essay, we refuse the stale opposition that characterizes many discus- 
sions of assessment. The terms of the debate are familiar to anyone who 
would purposefully pick up this volume: on the one hand, we have the specter 
of assessment as the dread realm of the numbers crunchers, the anti-intellec- 
tual adaptation of corporate models, the expensive and self-perpetuating 
industry that exists to fund layers of administrators and to quash intellectual 
freedom and meaningful scholarly endeavor. On the other, we have the specter 
of the violently defensive, autocratic faculty members who refuse reasonable 
explications, stonewalling clarity in their fear-based and ego-driven insistence 
on the unassessable alchemy they work in the classroom. Of course, there is 
some truth and much falsehood in both formulations. Assessment done with- 
out nuance is a waste of time at best and dangerous at worst, and faculty hos- 
tility to sharing clearly defined goals and criteria by which students are evalu- 
ated is unwarranted and unfair. 

So, as you gather, I am an assessment team player at Mills , 1 and yet, in 
writing this essay, I out myself as such in ways that, despite my self-conscious 
humor, feel risky to me. Part of that risk is endemic to the union/management 
structures of faculty/ administration that are simply part of higher education. 
Part of it, though, is that — as this volume takes as its topic — assessment runs 
into particular perceptual roadblocks within my discipline, that of literary 
study: roadblocks regarding questions of the measurable and the immeasura- 
ble, the joy factor, or the sublime, and our fear — a meaningful one — of reduc- 
ing our students’ work and our own to bite-sized enumerated chunks that sup- 
port only a pragmatic, unsubtle, fundamentally inappropriate and damaging 
way of approaching the study of literature, that study to which we have com- 
mitted our personal and professional lives and in which the vast majority of us 
engage with integrity and for the benefits such study brings to the lives of our 
students. The benefits exist at the level of what we often term the immeasura- 
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ble — sublimity and joy and perspective — as well as through students’ practice 
and movement toward what we often identify as the more clearly measurable 
applications of critical thinking and writing. I suggest that the judicious use of 
appropriate tools of measurement and the insistence on lucid transparent iter- 
ations of our expectations does not impede either benefit, but rather can help 
students and us better achieve both. 

Before you think I have lost my mind, or drunk too deeply from the assess- 
ment water cooler, I need to clarify that the following conditions predicate my 
commitment to assessment within my department and my institution: 1) the 
careful, nuanccd, and narrative nature of our work (its reliance on the eviden- 
tiary standard of our own discipline, not that of the business school); 2) the work 
and its terms are defined and implemented by the faculty and are not externally 
levied; and 3) the institution’s financial support of our projects and assistance in 
pragmatic help in translating our findings into the language and structures 
required of the institution and our accrediting board. I bemoan the proliferation 
of extra work and acronyms, and, when subsumed in cut-and-paste assessment 
criteria, I crack the occasional pained joke regarding our process of course-by- 
course assessment rubrics, unfortunately named CARP. However, our discipline- 
specific assessment of our literary studies courses has been an intellectually rig- 
orous process that is pliant and responsive enough to serve our students and our 
faculty and has increased, rather than diminished, our capacity creatively to 
teach students the skills they need to succeed as literature majors. 

Some History and the Nuts and Bolts of Our Assessment 

Our department first developed the English 1 0 “Introduction to Literature” 
course when we realized that, while the entire English faculty ranked close 
reading at or near the top of the list of skills our majors should demonstrate, 
we actually were unsure about where we explicitly taught the skills necessary 
to engage in close reading. While, as committed and excellent teachers, we all 
taught close reading and the introductory skills sometimes, in some places, we 
usually fit the subject in around the edges of our attempts to teach exploding 
canons, barely able to contain the work on content and context, let alone to 
provide systemic attention to the relation between reading and writing and the 
myriad incremental steps needed to write a strong, engaging, literature analy- 
sis. So we created a new class, a class in which our majors would be intro- 
duced to and practice the skills they needed to succeed in our major. 2 We were 
feeling pretty self-congratulatory about adding a lower division “service” class 
to our rotations out of the goodness of our teaching hearts, and all went fine 
through these rotations, or so we thought. 

One morning, about a year and a half after we began teaching the class, I 
received a puzzled call from the registrar’s office that a student was unable to 
add one of her courses, a course which she had been attending for the six 
weeks since school had been in session. In following through, it turned out that 
one of our majors had been attending two sections of “Introduction to 
Literature,” that she adored them both, and that nothing at all in the syllabus 
(other than the course number) or in the classroom experience, indicated that 
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these two classes were different sections of the same course. It seemed that we 
had created a terrific class, or a variety of classes, but it was self-evident that 
we needed to take a closer look at what we were actually doing in this class 
and why. So, I wrote a proposal to the provost’s office in which I asked for rea- 
sonable, if not at all handsome, funding to support three faculty members 
(myself and two other literature professors) in what is called a “full-loop assess- 
ment” that would run over two years. 3 Additional English faculty would be 
paid a small stipend to participate in less onerous iterations of the assessment, 
and then the full department would be consulted and have the opportunity to 
provide feedback/input because the course is required of all majors. Three 
tenured literature professors (myself, an eighteenth-century British literature 
specialist, Diane Cady, a medievalist, and co-author Ajuan Mance, an African- 
American literature specialist) spearheaded the assessment. We began by infor- 
mally talking to other faculty who taught the course (each of us had taught it 
once), collecting syllabi from our own department and from similar courses at 
other colleges, and making arrangements for the collection of student essays 
from each section of the class. 

We were ready to begin our formal work: we had to come up with a 
rubric by which we would assess the student papers (collected from each of 
the “Introduction to Literature” sections over the course of the year). First, we 
decided we would clarify what we thought the “Introduction to Literature” 
course should accomplish — what should students learn in this lower division 
introductory course? What were our overarching goals for student learning? 
We each came to that meeting with a list. Our combined lists constructed the 
Platonic ideal of Introduction to Literature courses — if our students could 
accomplish everything on that list, they would be happily succeeding in doc- 
toral programs by their sophomore years. Our list included, but was even 
longer than, the following: 

Students should: understand generic difference; be familiar with 
major trends in literary theory; be encouraged and strengthened 
as life-long learners and lovers of literature; read well; write clear, 
logical, argument-based interpretive essays; come up with engag- 
ing questions to drive essays; write clear and meaningful thesis 
statements; master syntax; develop style; choose and use quotes 
with grace; learn the tools of disciplinary research; engage in 
close readings of texts; engage in meaningful discussions; have an 
awareness of politics of canonicity; be familiar with major critical 
movements and debates in the field; master the MLA style sheet; 
read a variety of literary works from a variety of genres and 
styles and periods and authors from a variety of social positions; 
find their voices — oral and written; find joy and meaning; be 
strong peer editors; learn literary terms. (Saxton) 

All of these elements are indeed important to our students’ success in the field, but 
when we tried to figure out how we could teach them all, and how we could 
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observe and measure whether we were succeeding, we soon realized our hubris. It 
became clear why our different sections of the course were so, well, different, and 
how ill-defined our sense of what tiffs fifteen-week course really was. One class 
could not possibly do all, or even a majority, of tiffs teaching and learning work. 

Our next task was to come up with a better defined set of measurable goals. 
We each brought a refined goal set to another meeting and honed our list into 
the one we brought to the department at large for discussion (the meeting includ- 
ed all tenure-track professors as well as those adjunct professors who might teach 
the course). We asked one professor to serve as a formal facilitator so we would 
not veer off topic, and we had an intellectually and personally engaging two-hour 
meeting in which we seriously debated the role of the “Introduction to 
Literature” class, and thus, of course, the role of the English major, what is 
teachable, and what is not. We had the kind of discussion we almost never have 
as a group as we are usually preoccupied with staffing and budgets and so forth. 

Our list shifted in smart ways, and the department approved the following: 

English 10: Student Learning Outcome Goals 

• Students will learn how to write an effective literary analysis. 

• Students will learn the skills of close reading. 

• Students will become familiar with the genres and conven- 
tions of poetry, prose and drama. 

• Students will learn MLA documentation and the research 
methods of the discipline. 

• Students will become familiar with the language of the disci- 
pline (including critical terminology and concepts) and the 
critical issues and questions of the field. 

• Students will be exposed to the pleasures of critical reading 
and conversation. 

Then our summer work began. We had stacks of student papers to read, 
and we needed to have a means by which to measure them that would provide 
specific information (rather than the more general information from a simple 
letter grade). We needed a rubric. We looked at Introduction to Literature 
rubrics from other schools and at our own expository writing rubric. We tin- 
kered until we created a guide we felt was specific enough to help us and stu- 
dents, and flexible enough to allow for the subjectivity that is always at play in 
the response to narrative prose. We then gave the guide to our colleagues, made 
some minor adjustments based on their input, and came up with the following: 

The rubric for English 1 0 papers defines six skill-sets and includes 

four competency rankings. 

Skills: thesis; support; organization; style; use of Modern 

Language Association formatting/ citation, and mechanics. 

Levels of competency: accomplished (4); competent (3); devel- 
oping (2); and beginning (l). 4 
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We then spent a few long days reading and assessing piles of student essays 
(with grades and names of teachers and of students removed). Each paper was 
read separately by two of us. When we finished, we turned the piles of paper 
over to our assessment office, which tabulated the results. 

Our findings were more troubling than we anticipated. Our assessment 
revealed that students were not adequately performing in five of the six learn- 
ing outcome goals we had defined. For example, we found that only 13% of 
student papers demonstrated accomplishment in thesis; 62% were competent, 
28% developing, and 9% beginning. The course evaluations and movement of 
students up within the major supported that the sixth goal (that of the expo- 
sure to the pleasures of critical reading and conversation) was being well met. 
We realized that English 10, as the introduction to the major, needed to be 
refrained to focus on structured ways to ensure that our students gain the skills 
they need to succeed across and up through the major. 

As a result of our assessment, the major change we made was the nominal 
and thematic shift of the class from content (the literature we read) to skills 
(the skills we bring to the reading and discussion of and writing about that lit- 
erature). Our first act was to rename the course “Introduction to Literary 
Studies,” rather than “Introduction to Literature,” a shift that heralded the 
new focus of the course. 

We next turned to the creation of curriculum guidelines to provide a struc- 
ture for “Introduction to Literary Studies” courses; we realized we needed to 
create a document that successfully articulated the course goals upon which we 
had agreed as a department and that offered suggestions /guidelines about how 
to organize the course so that we could successfully teach to those goals. At this 
point, the work got sticky. I include our guidelines below and then some discus- 
sion of the whys and hows of their iteration. The following are the guidelines 
we adapted and distributed to all professors teaching the class: 

Curriculum Guidelines for Introduction to Literary 

Study: English 10 

Introduction to Literary Studies is a skill-based rather than con- 
tent-based course. 

The course focuses on teaching scaffold skills that will help stu- 
dents write effective literary critical essays. 

These skills include: 

1. Textual analysis/close reading. 

2. Appropriate use of literary terms and concepts. 

3. Thesis creation and development. 

4. Essay organization. 

5. MLA style and documentation. 

6. Research tools. 
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7. Avoidance of plagiarism. 

Course materials: 

All reading assignments must come from the required texts or be 

posted on Blackboard (e-reserve). 

1 . The Norton Introduction to Literature (most recent edition). 

2. The MIA Handbook for Writers of Research Papers (most recent 
edition). 5 

3. Grammar handbook. 

General information: 

1 . We explain the theoretical and practical structure of the 
English major and the role of English 10 in the context of 
the major. 

2. Syllabi clearly and succinctly indicate the purpose of each 
reading assignment (for example, introduction to characteri- 
zation, discussion of symbolism). 

3. Syllabi clearly and succinctly indicate the purpose of each 
writing assignment (for example, practice explication, differ- 
entiation between summary and analysis). 

4. Course information sheet includes department English 1 0 
course goals, grading breakdown, and rubric in addition to 
regular required information. 

5. Each course includes a library information session, led by a 
librarian, on literary research methods (to be scheduled by 
individual professors with the library). 

Reading: 

1 . Students will read poetry, short fiction (not novels), and one 
play. 

2. Because the course is skill-based, readings should not be 
excessive, and each reading should serve a clearly articulated 
purpose. 

Writing: 

Essays 

1 . Students will write three graded essays (with built-in draft 
workshop and revision processes) of no fewer than three and 
no more than seven pages each. 

2. Student essays should not incorporate outside critical works. 

3. Student essays should accurately document primary textual 
source(s). 

4. The course will explicitly familiarize students with the tools 
necessary to write strong essays (see enclosed sample hand- 
outs on thesis generation/reverse-outlines/integrating pri- 
mary texts). 
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Homework/In-Class Writing Assignments 

1 . Students will have a variety of graded opportunities to prac- 
tice the skills required to write successful literary analytic 
essays. 

2. These assignments should provide helpful scaffold opportuni- 
ties; they are not essays. 

3. These homework and in-class assignments should be short 
and clear; for example, a quiz or a homework assignment on 
summary versus analysis, or short explication. 

4. Students must complete a homework assignment (such as a 
brief annotated bibliography or a research scavenger hunt) 
that introduces them to library and on-line research methods 
in literary study. 

Grading: 

1. Professors fill out a rubric sheet for each graded student essay 
and return it with a final letter grade and narrative comment. 

2. Grading breakdowns fall within the following guidelines: 

a. Three essays for a total of 60%. 

b. Scaffold-skill based assignments for a total of 30%. 

c. Participation and Attendance for a total of 1 0% (clearly 
define participation and record attendance at each session). 

In addition to regularizing grading standards (by using our newly minted 
department rubric), the guidelines regularize the number and weighting of 
assignments (areas which previously were entirely up to professorial discretion). 
Before this assessment, the course’s only clear and shared requirement was 
that it covered drama, fiction, and poetry. As a department, it was important 
to us that we strike a balance between creating parameters and allowing for 
professorial choice. For example, we agreed that students should all use the 
same main primary text, but we were leery of limiting professors’ control over 
the course reading content. We chose the Norton Introduction to Literature for its 
wide scope of authors, periods, and genres. (Over the course of the following 
year, we found that the cost and physical size of the book were prohibitive, 
and we adopted the shorter edition.) 

Our decisions about content and about the number and type of assign- 
ment were the result of hard won and complex conversations in which we 
each became aware of our own stakeholding status in regard to our comfort 
levels (what assignments we like to give, what we do not, what we want to 
read, what we do not). For example, our discussions about whether or not 
“Introduction to Literary Studies” should require a Shakespeare play were 
contested and tense, although our agreed focus on skills allowed us to decide 
that we should not use this course as a content band-aid for our loosening up 
on our major requirement of a Shakespeare course. We kept being seduced by 
content: of course they should read Shakespeare, and they should read a 
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diverse variety of authors, from a variety of periods, and perspectives, and for- 
mal styles, etc. And we would end up right back in content land, right back 
where our surveys and genre and period and topical courses were supposed to 
be. If we wanted Shakespeare, we needed to require that elsewhere, not in our 
introductory skills class. The same was true regarding coverage of formal and 
thematic and author-specific texts. 

Instead, we made clear that the course must “explain the theoretical and 
practical structure of the English major and the role of English 10 in the con- 
text of the major” — in other words, that the class would make transparent the 
major’s requirements and how they fulfill our mission of teaching diversity, 
period, genre, nation, and form, as well as depth and breadth. We asked pro- 
fessors to explain notions of literary canonicity and to talk about the ways the 
structure of our major engages with and takes a position on debates on canon- 
icity. We decided that professors could best teach this course using readings 
with which they were comfortable and that we did not need to use the course 
guidelines as a stick with which to beat our colleagues with our own prefer- 
ences. We had to conclude that we trust one another to teach with integrity 
and to make content choices that make sense. Our refusal to legislate content, 
however, led to our decision to require that each reading response have a clear 
indication of its purpose (a few words or a clause), of what the professor want- 
ed it to “do” in the class. As a teacher, I hate writing these notations. However, 
this requirement has proven a truly wonderful intellectual challenge in that I 
have to articulate, to myself and to my class, why we are reading Percy Bysshe 
Shelley’s “Ode to the West Wind” (it exemplifies the ode form), why we are 
reading James Baldwin’s “Sonny’s Blues” (a means to discuss symbolism in fic- 
tion), why we are reading Raymond Carver’s “Cathedral” (the development of 
character) — and in all three cases, because these are works I know well and 
love and find teachable (which I tell them). I find that my evaluations, already 
good, skyrocket when I make explicit such pedagogical drives. 

In regard to assignments, we decided revision and opportunities to prac- 
tice the skills we demand are essential. While we all believed this, none of 
our previous “Introduction to Literature” syllabi actually demonstrated this 
belief. My earlier syllabus included four or five essays, a number of undirect- 
ed responses to the readings, and very little overt teaching of the writing 
skills I so pride myself on teaching well. Now, we all require only three for- 
mal essays (each with multiple draft steps). We agreed that we do not want 
students to use critics at this stage of their writing since what they need to 
work on in their papers is the construction of their own interpretive argu- 
ments. However, we want them to have some discipline-specific introduction 
to the library (something we realized we had never offered our students for- 
mally). So we require a reference library workshop led by a librarian; that 
session is followed by a scavenger hunt assignment that requires students to 
find, but not to read, a number of sources and materials. We ask that each 
professor remain true to the “sort” of materials we ask students to find, but 
to use her own reading materials as the basis for the hunt. For example, my 
assignment asks students to find, among other things: a critical book on 
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Hamlet that was published before 1950; a scholarly article written within the 
past three years that approaches Hamlet through a feminist lens; a popular 
review of a production of Hamlet ; the complete Oxford English Dictionary defi- 
nition of “nunnery”; and a critical article that looks at flower images in the 
play. We also require that professors offer a number of opportunities to 
practice the writing skills needed to craft a successful literary analysis; for 
example, that students practice close readings, summary and paraphrase 
activities, and thesis generation. We decided that professors were best posi- 
tioned to choose the assignments that their students needed and so we left 
the specifics up to each teacher. 

Full Loop: Teaching Our New Class 

After the department had taught a full year of “Introduction to Literary 
Studies” courses according to our revised curriculum guidelines, we asked each 
professor to collect student papers for review the following summer. That sum- 
mer we returned to them with our rubrics. We also asked each professor to give 
us informal feedback on what worked and what did not. The results (both 
informal and formal) were informative: there was a 10% jump in the category 
of “accomplished” theses and a 7% jump in that of “competent” ones, for 
example. Overall, essays were more accomplished and better organized with 
stronger support and much more accurate in their use of MLA citation than in 
our initial assessment. We had a jump in accomplished (the strongest) papers 
overall, and a decrease in beginning (or the weakest) ones. Student evaluations 
were up in the classes and professors’ qualitative feedback was positive. 

Intra-dcpartmental buzz around the subject of student preparedness sug- 
gests that the “Introduction to Literary Studies” assessment has made a lasting 
impact on the quality of student work across the department. Faculty across 
the department comment that students who have completed our revised 
“Introduction to Literary Studies” course demonstrate greater confidence and 
skill in the areas of critical thinking and analysis, skills reflected in their 
engagement in class discussion as well as in their written work. The “Introd- 
uction to Literary Studies” assessment precipitated changes in the structure of 
the course that have enhanced its effectiveness as a gateway into the discipline. 

Part 2: The Future of Assessment in the Major: Literature, 
Diversity, and Measuring the Unmeasurable 
Ajuan Maria Mance 

In the previous section, Kirsten T. Saxton describes her thoughts on the 
Mills College “Introduction to Literary Studies” assessment, as well as the details 
of our process and its outcomes. In this portion of our essay, I will address some 
of the ways that this work has led not only to my measured enthusiasm for the 
possibilities of nuanced assessment of introductory level, skills-based English 
courses, but also to my budding interest in the ways that assessment could sup- 
port and enhance the engagement of both students and faculty with diverse lit- 
eratures and methods of analysis. I am especially interested in considering the 
role that assessment could play as literature departments move beyond conccp- 
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tions of diversity education (also called multicultural education), in which stu- 
dents’ exposure to women writers, queer writers, and writers of color is primari- 
ly concentrated in courses structured around those sub-specialties, and toward a 
more inclusive vision in which diverse texts, theoretical frameworks, and modes 
of analysis are incorporated throughout the curriculum. 

When I was asked to participate in the assessment of our department’s 
gateway course, my public willingness was tempered by a great degree of pri- 
vate skepticism. An English professor since 1995, my years in the classroom 
have been both intellectually and personally rewarding, and I began the 
assessment process unconvinced of its capacity to account for the wonderful 
alchemy by which a carefully crafted syllabus, the engagement and spontaneity 
of in-class discussion, and individual reflection that takes place outside the 
classroom can profoundly alter a student’s relationship to both text and con- 
text — the social, cultural, and theoretical frameworks that shape both the liter- 
ature in question and its interpretation. I was apprehensive about how the 
established language and protocols of assessment would account for those 
flashes of comprehension in which a student’s encounter with a text, perspec- 
tive, or idea causes a shift in what the poet Audre Lordc calls “the quality of 
light” by which she sees a particular text or its themes. The flashes are integral 
to the postsecondary experience (36). 

In retrospect, I realize that most of my misgivings arose from my concern 
about whether this process would transmit our department’s commitment to a 
diverse curriculum that utilizes, alongside more mainstream and canonical 
works, the texts of women writers, queer writers, writers from working-class 
and poor backgrounds, and writers of color. And my concern was not only, or 
even primarily, related to the “Introduction to Literary Studies” course itself, 
but to those content-oriented surveys and upper-division seminars for which it 
serves as preparation. In the end, however, the “Introduction to Literary 
Studies” assessment did more than demonstrate for me the usefulness of 
meaningful, nuanced faculty-driven assessment and course revision. When I 
apply what I learned as a participant in this process to my experience as a pro- 
fessor of African American literature, I begin to understand how assessment 
can benefit the teaching of marginalized literature and how the assessment 
process can point to the possibility of both naming and measuring some of 
those crucial outcomes of undergraduate education to which my colleague 
and co-author alludes in her reference to “those questions of measuring the 
unmeasurable.” Just as the “Introduction to Literary Studies” course itself 
serves as a gateway for Mills College undergraduates into the English major 
and into a fuller and richer engagement with critical analysis, so too has the 
“Introduction to Literary Studies” assessment become a gateway for me, into 
a new and compelling way of thinking about both the transformative nature 
of the literature classroom and the pedagogical demands that define my spe- 
cialty area of African American literature. These two concerns intersect in 
ways that expose a new layer of pedagogical practices and intellectual out- 
comes to evaluate and explore. In considering how the assessment process and 
the field of US Black literature might shape and transform each other, I can 
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imagine a not-too-distant future in which the goals and criteria established for 
measuring student outcomes in African American and other marginalized lit- 
eratures might suggest a means to name and measure those very qualities of 
magic or alchemy that are so fundamental to both students’ and instructors’ 
classroom encounters with the literary text. I am speaking, quite simply, of the 
ways that goals and assessment criteria developed to strengthen and enhance 
the teaching of identity-based literature fields may point to the very real possibility 
of transforming the teaching of minority literatures across the English curriculum. 

When I think about how the alchemy of college teaching works in my 
own literature courses, my thoughts turn to those moments in which students’ 
interactions with the text, with me, or with each other accomplish that which I 
often think about but fail to express or deliberately to plan for in my teaching: 
that is students’ experience of the close examination of a text or a body of 
texts as precipitating a shift in their thinking about both the assigned class- 
room texts and the socio-political and cultural environment in which they (the 
students) read and understand those texts. I believe that it is possible and even 
essential to articulate much more specific goals and criteria for measuring a 
course’s success in linking the students’ analyses of assigned literary texts to 
their interrogation of social, economic, cultural, and political practices, beliefs, 
and assumptions that shape both the text itself and their engagement with it. 

I will use the field of US Black literature, my own area of specialty, as my 
first example to tease out some of the possibilities I see for assessment as a 
tool. What might be gained by developing a set of criteria to measure not only 
students’ comprehension of the breadth and diversity of US Black literature 
before the Harlem Renaissance, but also the impact of, for example, an 
African American literature survey course on students’ understanding of iden- 
tity categories and subject positions, relationship and settings above and 
beyond those depicted in the assigned readings? Consider, for example, that 
one of the most dramatic and effective moments in any of my African 
American literature classes takes place when I call students’ attention to the 
ways that many US Black writers highlight the African American gaze. 6 I 
often guide the class through a close reading of a passage depicting the white 
subject as other, in effect facilitating my students’ encounter with whiteness as 
objectified (and, at times, exotic) other, often for the first time in their lives. 
This decentering of whiteness can feel uncomfortably destabilizing for under- 
graduate and graduate students alike, across ethnicities. 7 And yet this is an 
essential discomfort. The displacement of whiteness is both a means for and a 
side-effect of writing Blackness into the center. This concept of decentering of 
whiteness and the accompanying practice of re-examining assigned readings 
in African American literature for the ways that their privileging of the Black 
gaze rearranges conventional (and racializcd) conceptions of same versus 
other is, thus, critical to the teaching of US Black literature in the same way 
that a basic knowledge of the conventions of Elizabethan English is critical to 
the teaching of Shakespeare. The capacity to analyze an African American 
text in this manner must therefore be indicated in any department-wide or 
course-specific assessment efforts as a goal, with criteria that can be measured 
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to determine the success or failure of a department in accomplishing this goal. 

My home institution of Mills College was one of twenty-eight California 
colleges and universities selected to be part of the James Irvine Campus 
Diversity Initiative (CDI). 8 Among the happy results of our participation in 
this initiative have been the development of a number of new courses on 
issues related to underrepresented racial and ethnic groups as well as the 
revamping of a number of more general courses (some at the introductory 
level) to make them inclusive of issues and texts related to underrepresented 
groups, a move which brings me back to the concept of decentering whiteness 
and its integral relationship to the teaching of African American literature. 
This diversification of the curriculum is a wonderful development, as it means 
that work by and related to the conditions, history, and experiences of people 
of color, women, and sexual minorities is fast becoming interwoven into our 
course offerings in ways that feel both pcdagogically sound and intellectually 
true. But simply to include literary and artistic works and scholarship by peo- 
ple of color and other marginalized groups is not enough. 

This revision and expansion of primary textual content must be accompa- 
nied by the incorporation of the scholarly terms and concepts, theoretical 
frameworks and analytical tools most effective at exposing and highlighting the 
counter-hegemonic themes and ideas often advanced in these bodies of work. 
Assessment could be a wonderful tool to ensure that students are exposed to 
and understand the major themes and techniques utilized in literature written 
by people of color, queer writers, and women, and are familiar with the goals 
of those scholarly modes of analysis developed in response to this primary lit- 
erary work (theoretical lenses such as feminist theory, Black feminist theory, 
queer theory, Xicanisma, and others). 

The pitfalls of teaching marginalized literatures without drawing on these 
modes of analysis are clear. To examine, for example, Gloria Anzaldua’s 
Borderlands without the benefit of Chicana feminism’s embrace of intersection- 
ality and its related rejection of white supremacy leaves the text vulnerable to 
readings that would cxoticize the writer’s perspectives, positionality, and/ or 
language(s).9 Such a reading would overshadow the text’s explicit challenge to 
whiteness and to English as normative standards. Assessment of student out- 
comes in courses that draw some of their readings from Chicana/o, queer, or 
other minority and women’s literatures must include as a goal or set of goals 
(with measurement criteria) students’ engagement with some of the ways that 
these bodies of work interrogate those power relations that have contributed 
to their marginalization. 

I am, admittedly, only at the beginnings of my thinking about how assess- 
ment could serve to make concrete the ways that a more diversified syllabus 
enhances students’ knowledge of literature; and at this point, when it comes to 
the specifics of how to define and measure students’ engagement with writings 
by minorities and women, I have more questions than answers. In an American 
literature survey course, for example, would it not make sense to add as a goal 
of the course that students be exposed to modes of literary analysis that take up the spe- 
cific questions of power, aesthetics, and meaning raised in the literatures created by and about 
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marginalized groups? Would the corresponding criteria for measurement require 
students to demonstrate an understanding of the critical issues and concepts 
addressed through such approaches? Our survey courses often make use of 
exams, which might provide the desired fit for ascertaining undergraduates’ 
basic comprehension of the critical issues and terminology associated with dif- 
ferent theoretical methods. Might, then, the course guidelines for the lower- 
division survey advocate exams with short answer definition questions as the 
tool of choice for measuring students’ competency level around some of the 
key terms and themes addressed in minority, queer, and women’s literatures? 

At the upper-division level, goals and criteria that prioritize students’ 
engagement with the counter-hegemonic frameworks, effects, and structural 
inversions of marginalized literatures should prioritize and measure familiarity 
with those critical approaches that bring these bodies of work into focus. If the 
lower-division surveys might set as an outcome students’ exposure to the critical 
issues and terms that define these modes of inquiry, then would it not be rea- 
sonable to expect an advanced undergraduate English major to 1) demonstrate 
her understanding of the theoretical frameworks that explicate certain minority 
literatures’ aesthetic and thematic distinctions; and 2) demonstrate appropriate 
application of the theoretical concept or method to a relevant literary text? At 
the upper-division level, the literary essay is the primary means by which facul- 
ty gain an understanding of their students’ grasp of academic concepts and 
materials, and I can easily imagine as an outcome that students demonstrate the 
ability to craft a thesis that is rooted in a minority-based theoretical framework. 

From the focus on student outcomes, my thinking moves to the question of 
course guidelines for faculty, a phenomenon whose embrace seems to be inversely 
related to its proscriptiveness. How detailed could such recommendations 
actually be? Would it be possible or appropriate, for example, to set as a goal 
that students in classes that feature texts transcribed from, or influenced by, 
the African American oral tradition would learn to apply Henry Louis Gates, 
Jr,’s concept of signijyin(g) to close readings of appropriate texts? 10 Gould a 
department establish as a guideline for a course in women’s literature that the 
course include at least two theoretical frameworks originated by and for 
women writers of color? Would it be overstepping a boundary to recommend 
that those frameworks be Black feminist thought and Xicanisma? What if, in 
the process of assessing an introductory-level queer literatures course, the eval- 
uating faculty set as a guideline that instructors teach at least two texts and 
one theoretical framework addressing the critical issues and terms specific to 
lesbian literature, gay men’s literature, and trans literature, respectively, as well 
as one queer non-white perspective? 

Whatever its eventual role in addressing these issues and needs, assessment 
will be a critical part of ensuring that the strategies and theoretical approaches 
developed to interrogate the counter-hegemonic themes and goals of margin- 
alized literatures become (and remain) an integral part of the English literary 
studies curriculum. Assessment could become a vital player in efforts to further 
integrate into the teaching of marginalized literatures those theoretical 
approaches and scholarly methods that highlight, rather than obscure or con- 


272 



FROM SKEPTICISM TO MEASURED ENTHUSIASM 


tradict, their social justice aims. If we value as a student outcome the compre- 
hension of the strategies by which queer texts challenge heteronormativity or 
an African American text challenges white supremacy, or a feminist text chal- 
lenges patriarchy, then might we also value as an outcome students’ exposure 
to those critical frameworks created to highlight, interrogate and critique the 
means by which those texts pursue these and other related goals? This last 
question points to the range of possible roles for assessment in English depart- 
ment efforts to integrate more fully diverse texts and approaches throughout 
the curriculum. I am excited about the potential of increased attention to the 
specific pedagogical demands of minority and women’s texts to transform not 
only how we think about teaching literature, but how we think about evaluat- 
ing and measuring the effectiveness of literature instruction. 

Putting the Bogie to Bed: Our Assessment Coda 
Kirsten T. Saxton and Ajuan Maria Mance 

Our introductory course assessment had a ripple effect throughout the lit- 
erature curriculum, and at the time of this writing, in the spring of 2010, we 
find ourselves at an exciting point in the growth and development of our 
major. With a firmer foundation (in the form of our revised gateway course), 
we are now looking toward the assessment and potential reconfiguration of our 
surveys and capstone. This time, though, the successes of the “Introduction to 
Literary Studies” assessment have replaced most of our skepticism and appre- 
hension with curiosity, engagement, and, even traces of enthusiasm. 

One of the bogies that haunted our attitudes toward the assessment man- 
date was our fear of the conversations that assessment would require within 
our department. Not only did we fear that agreement on goals and criteria 
would be impossible or reductive, we worried about the effect that the depart- 
ment’s discussions about assessment might have on our views of one another 
and our ability to work together, that having to be so blunt about what we 
value and why would reveal untenable differences that would fracture our gen- 
erally congenial relations. In fact, our assessment-driven discussions about our 
differences of opinion and our preferences, our particular specialties and styles 
have created a more intellectually exciting and cohesive department at the 
level of both curriculum and community. As this co-authored essay demon- 
strates, our work on assessment within the major has led not to a disruption of 
our collcgiality and intellectual continuities but a deepening of them. 


NOTES 

1 I led the first assessment at our campus (our first-year composition 

course), and the first within our department (Introduction to Literature) 
and beginning in summer 2010, a third (our MA capstone). 

~ Beyond English 10, our major requires three survey courses, Shakespeare 
or the Bible as Literature, six upper division topical/ genre/period courses, 
and a senior thesis. 
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^ A “full-loop assessment” in this case means that we do the following steps: 
1) set achievable goals for student learning; 2) define those goals in ways 
that are observable/measurable; 3) develop a rubric or criteria for meas- 
urement of the student achievement of these goals; 4) collect student work 
and measure its achievement of each of the goals; 5) analyze and reflect 
on the results; and 6) decide if we as a department need to change things 
in our programmatic structure (of the major, a “type” of course, such as 
surveys, introduction to literature, senior seminars, etc.) in order for stu- 
dents to achieve better the outcomes or goals we want. Make those 
changes. Rinse and repeat for a full loop. 

^ See Appendix A for the rubric. 

^ We added the MIA Handbook because of its comprehensive citation mate- 
rial and because we believed it a necessary resource for all of our majors 
throughout their completion of the degree. 

6 My reference to the African American gaze is rooted in the more familiar 
theoretical concept of the male gaze and refers to the perspective from 
which the Black subject perceives. Feminist film theorists such as Laura 
Mulvey conceptualize the male gaze as one rooted in and manifesting a 
patriarchal power that locates and identifies the female as objectified 
other. African American writers frequently invert the asymmetrical power 
relations that define most US fiction (in which white characters engage 
the Black figure as the exotic) and offer a narrative rooted in the Black 
characters’ experience of themselves as center and white individuals and 
identity categories as the non-normative other. 

^ In my classes, the greatest resistance has come from those students of 

European descent whose experience of their own whiteness is conceptual- 
ized as the absence of race, and those African American students whose 
understanding of their own subjectivity is rooted in their belief that 
Blackness in the US is, by definition, incapable of marginalizing the iden- 
tity and experiences of others. 

^ Initiated in 2000, The James Irvine Campus Diversity Initiative was a six-year 
program of grants and other forms of support for diversity programming on 
the campuses of twenty-eight independent colleges throughout California. 

^ Intersectionality, a core concept in US literature by women of color, is 
based in the notion that racial, ethnic, gender, class identities each inter- 
sect with and inform an individual’s experience of his or her identity. An 
early text that exemplifies this idea is the slave narrative, Incidents in the Life 
of a Slavegirl (1861), by Harriet Jacobs, a nineteenth-century African 
American woman writer. The term intersectionality (also called multiple 
identities and interlocking oppressions) applies to the way that Jacobs’ 
identity as a woman is inextricable from her identity as Black, with each 
of her identities shaping not only the way that she experiences the other, 
but also the way that others experience her (as simultaneously Black and 
woman). Add to her race and her sex the fact that, during the events she 
recounts in her narrative, she was also enslaved, and her experience of 
and visibility as both Black and a woman is further complicated by the 
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question of class. For further discussion and examples, see “A Black 
Feminist Statement,” by the Combahee River Collective, in All the Women 
are White, All the Blacks are Men, But Some of Us are Brave, edited by Gloria T. 
Hull, Patricia Bell Scott, and Barbara Smith; and Woman Warrior: Memoirs 
of a Girlhood Among Ghosts, by Maxine Hong Kinston. 

In The Signifying Monkey , Henry Louis Gates, Jr. uses the title of a traditional 
African American vernacular poem as the basis for his introduction of a lit- 
erary theoretical concept, called “signifyin(g).” The terms refers to what 
Gates argues is a uniquely African American strategy of engaging past 
texts — stories, songs, rhymes, and other expressive forms — through imita- 
tion, amplification (of themes, characters and character types), and reversal. 


APPENDIX A 

English 10 Essay Assessment Guide 


Overall 

Score 

4. Accomplished 
(Shows skill; many 
strengths present) 

3. Competent 
(On balance the 
strengths outweigh the 
weaknesses) 

2. Developing 
(Strengths and need 
for revision are about 
equal) 

1. Beginning 
(Need for revision 
outweighs strengths) 

Thesis 

30% 

Clear and original thesis 
of appropriate scope 

Clear but less original or 
refined thesis 

Thesis present but needs 
work in clarity, originality, 
or scope 

Thesis weak or inade- 
quate: absent, unclear, or 
inappropriate in scope 

Support 

25% 

Argument is well devel- 
oped; paper provides 
appropriate, carefully 
analyzed supporting tex- 
tual evidence 

Argument is reasonably 
well developed and sup- 
ported, but needs more 
carefully analyzed textual 
evidence 

Evidence and/or analysis 
weak 

Almost no appropriate 
evidence and/or analysis 

Organization 

25% 

Consistently logical pro- 
gression of ideas and use 
of logical transitions 

Generally logical progres- 
sion of ideas and generally 
logical transitions 

Confusing progression as 
a whole and/or within 
paragraphs 

Lack of logical progres- 
sion of ideas and lack of 
transitions impede 
understanding 

MLA 

5% 

Accurate use of MLA 
documentation 

Minimal errors in MLA 
documentation 

Significant errors in MLA 
documentation 

Inadequate use of MLA 
documentation 

Style 

5% 

Successful tone, varied 
sentence structures, clear 
and confident prose, and 
appropriate and accurate 
use of literary terms and 
concepts 

Reasonably successful 
tone, varied sentence 
structures, clear prose 

Less successful tone, less 
varied sentence struc- 
tures, less clear prose 

Tone, sentence structure, 
and prose style impede 
paper 

Mechanics 

10% 

Essay demonstrates mas- 
tery of standard conven- 
tions of spelling, gram- 
mar, syntax, and punctu- 
ation 

While there may be minor 
errors, the paper follows 
standard conventions of 
spelling, grammar, syntax, 
and punctuation 

Frequent errors in 
spelling, grammar, syn- 
tax, and punctuation are 
distracting 

Writing contains numer- 
ous errors of spelling, 
grammar, syntax, or 
punctuation that interfere 
with comprehension 


Source: Mills College, Department of English, Summer 2009. 
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A CAUTIONARY TALE ABOUT SYSTEM-WIDE 
ASSESSMENT IN THE STATE UNIVERSITY OF NEW 
YORK: WHY AND HOW FACULTY VOICES CAN AND 
MUST UNITE 

PAT BELANOFF AND TINA GOOD 


Assessment is not evaluation, nor is it competition. Assessment is 
a process, first and foremost, for understanding and improving 
student learning. 


SUNY Provost’s Advisory Task Force on the 
Assessment of Student Learning Outcomes 


Fall 1995 

On December 1, 1995, the Board of Trustees of the State University of New 
York (SUNY) presented the document “Rethinking SUNY’ to the thcn-Governor 
of New York State, George E. Pataki. This document was created in response to a 
call from the New York State Legislature requesting a “multi-year, comprehensive, 
system-wide plan to increase cost efficiency” (qtd. in Board of Trustees of SUNY, 
“Rethinking SUNY” 1)T The preface notes that “as appointed overseers of the 
State University of New York, the Board of Trustees has a continuing responsibili- 
ty to assess its use of the state’s investment and to seek positive changes to ensure 
that we are delivering the most effective services to the taxpayers and the students 
of the State University of New York” (1). Within this context of “cost efficiency,” 
one of the seven propositions put forth is the following: “Clearer academic stan- 
dards and better means for measuring performance are central to increasing 
accountability” (2). What follows is a detailed and documented account of actions 
resulting from this 1 995 document. The undercurrent here (as in almost all discus- 
sions of standardized assessments) is the tension between assessment to improve 
student learning and assessment for accountability. 

Fall 1998 

As a follow-up to “Rethinking SUNY,” in December 1 998, the Board of 
Trustees passed Resolution 98-241- mandating the establishment of a general 
education requirement for all students working toward a bachelor’s degree at 
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all of its campuses. To meet this charge, the Provost created an Advisory Task 
Force on General Education whose membership included campus officers, fac- 
ulty, and students. In the document, “Implementation Guidelines for State 
University of New York Baccalaureate Candidate General Educational 
Requirement,” this Task Force mandated ten “knowledge and skill” areas and 
two “competencies” for all general education programs on all system campus- 
es. 3 Furthermore, the Task Force required all campuses to include “assessment 
as a necessary component of their General Education implementation plans.” 
To fulfill this requirement, campuses were mandated to: 

• Review their entire General Education programs periodically to 
evaluate and strengthen performance; 

• Establish assessment programs for the specified student learning 
outcomes; 

• Derive evidence from the application of such assessment pro- 
grams to show that the intended learning outcomes are being 
achieved; and 

• Use the results of assessment programs to improve the quality 
and effectiveness of General Education programs. 

Fall 1999 

In response to the General Education Task Force’s recommendations for 
general education assessment, the Provost established an Advisory Task Force 
on the Assessment of Student Learning Outcomes. This Task Force consisted 
of members from all segments of the university: campus presidents, chief aca- 
demic officers, faculty, and students. As articulated in the “Report of the 
Provost’s Advisory Task Force on the Assessment of Student Learning 
Outcomes and Guidelines for the Implementation of Campus-Based 
Assessment in the State University of New York,” its charge was to 

examine and discuss the issues involved in implementing under- 
graduate student learning outcomes assessment today — specifical- 
ly, in the context of a large and diverse university system such as 
the State University of New York. (3) 

And to make recommendations regarding 

a process for assessing student learning outcomes and intellectual 
growth in General Education and the Major that will provide the 
faculty and academic leadership with an important and effective 
way of improving the quality of undergraduate education, and 
the University with a coherent and meaningful longitudinal data 
base with which to be accountable to its stakeholders. (4) 

The Task Force made the assumption that assessment could work toward 
improving teaching and learning while simultaneously providing information 
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for purposes of accountability. 

In order to achieve this dual mission, the Task Force recommended two 
levels of assessment: 

The SUNY Assessment Initiative should consist of both campus- 
based and University-wide strategies, with campus-based assess- 
ment focused primarily on program improvement and University- 
wide assessment used primarily to serve accountability and advo- 
cacy functions. (29) 

What this statement came to mean was that individual campuses could devel- 
op and implement their own assessment plans for the twelve general education 
areas as specified by the Task Force and for the major. General education 
assessment plans were to be approved by another committee established by the 
Provost’s office: the SUNY General Education Assessment Review (GEAR) 
committee. This committee was chosen to represent all levels of the SUNY 
system: university, college, and two-year institutions. Interestingly, the Task 
Force did not recommend university- wide assessment of academic majors, 
although it did not state the reason for this decision (3-4). 

In addition, a second-level of assessment was to be instituted that would 
require SUNY institutions to implement some sort of assessment tool (such as a 
standardized test) to measure students’ learning outcomes in all twelve general 
education areas for the purposes of comparison, accountability, and reporting. 

Although the Task Force Report made it clear that there were to be two 
levels of assessment developed for the general education outcomes, this duality 
was not widely recognized by the faculty. Instead, the general belief was that 
campus-based assessment plans had to be developed, and as long as a campus 
developed these plans in good faith, there would be no mandate to implement 
a standardized university-wide assessment plan. In fact, it was understood that 
the development and implementation of the local plans would be a process, 
and as these plans were put into play, they would be honed to meet the needs 
of the faculty, students, administrators, and System Administration. 

Summer 2003 4 

Many concerned faculty worked on developing plans without any added 
compensation. Quite a few campuses had already sent their proposed plans 
and schedules to the GEAR committee.'’ Members of this committee, working 
in small groups of three, examined these plans and cither returned them to 
the campuses with suggestions for revision or sent them, with the committee’s 
recommendation, to the Provost’s office for final approval. Many campuses 
had even begun administering the plans and reporting results to System 
Administration. Therefore, when the Board of Trustees issued a resolution on 
June 17, 2003 (“Resolution on System-wide Assessment”), demanding univer- 
sity-wide and standardized assessment in the twelve general education areas, a 
great sense of shock and betrayal swept through SUNY’s colleges and univer- 
sities. Throughout the following year, faculty and administrators reeled from 
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the aftershocks of the Board’s resolution. Faculty response, by way of their 
governance bodies, was swift and unequivocal. 

SUNY’s statewide faculty governance consists of two bodies. One body is 
made up of representatives from the state-operated colleges, that is, all the col- 
leges and universities within the system except for the community colleges. 
This body is known as the University Faculty Senate (UFS). The other body is 
made up of representatives from all the community colleges within the system 
and is known as the Faculty Council of Community Colleges (FCCC). 6 

The FCCC was first to respond. On July 9, 2003, Kimberly Reiser, 
President of the FCCC, issued a letter of response in which she stated: 

I am writing on behalf of the faculty of the community colleges 
of the State University of New York to voice our determined 
opposition to the resolution adopted by the Board of Trustees on 
June 1 7, 2003 for the implementation of system-wide assessment. 

Our objection to this resolution is twofold. First, we are and have 
always been opposed to system-wide common metrics to assess 
learning; we believe they will circumvent the campus-based 
assessment process that the faculty and administrators of this 
University have worked so hard over the past two years to devel- 
op and implement. Second, we have grave concerns about the 
process by which this resolution was adopted. 

Reiser went on to admonish the Board for failing to follow the by-laws govern- 
ing the issuance of such resolutions and for abruptly ignoring negotiations 
already taking place by faculty and System Administration. She concluded 
with the request that the resolution be rescinded. The request was denied. 

Fall 2003 

On September 26, 2003, GEAR drafted a memorandum of understand- 
ing (MOU) between SUNY Faculty Governance and System Administration 
regarding the “Development and Pilot Process for Value-added Assessment.” 
This memorandum, a result of the furor caused by the June 2003 resolution, 
attempted to define “value-added assessment” and called for the formation of 
the “Value-added Assessment Development Group (VADG)” whose charge 
was to develop 

the assessment instrument(s) to be used, the administrative proce- 
dures, issues related to student motivation, and the reporting pro- 
tocol to be followed. The assessment should be made at two dif- 
ferent points in time so as to permit the determination of the 
growth in learning that has occurred (“value-added”) and should 
be accompanied by an assessment of the level of student engage- 
ment in academic activities that tends to result in higher levels of 
academic achievement. 
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This charge specifically mentions learning outcomes in “Mathematics, Basic 
Communication, Critical Thinking (Reasoning), Information Management, 
and the understanding of the methods scientists and social scientists use to 
explore phenomena.” Interestingly, the list does not include humanities, the 
arts, or foreign language, although outcomes in these areas are a part of the 
original list of general education areas. Was this an oversight or deliberate? 
The reason for the omissions is not stated. 

This MOU concluded with a listing of “faculty concerns.” Among these 
concerns were the following: 

• System-wide assessment will ultimately lead to inappropriate 
comparisons between institutions with vastly different missions. 

• System-wide assessment will include the use of standardized tests 
or other common measures which will have the effect of forcing 
faculty to teach to the test. This would standardize our curricula 
and inhibit or remove the diversity which is a quality indicator for 
a system of higher education. 

• Assessment ought to be campus-based, rather than System-wide. 
Further, campuses have in good faith developed campus-based 
assessment plans for general education. Now, before these plans 
have had the chance to be implemented, this additional layer of 
assessment is being urged. This sends the message that the cam- 
pus-based plans were inappropriate or not rigorous enough. 

• Presently, most campuses are having real difficulties as a result of 
the financial situation faced by New York State and the nation. 
System-wide assessment would represent one more unfunded 
mandate that would divert scarce resources from the faculty to 
another System mandate. 

System Administration agreed to address the concerns that were listed in the 
MOU, although the specifics were left undefined. The MOU was distributed 
to the delegates of the Faculty Council and the Faculty Senate with the 
Chancellor’s request that they endorse it. They did not. 

In October 2003, the FCCC and the UFS passed resolutions opposing the 
Board’s June 2003 resolution. On November 1, 2003, the Student Assembly 
passed a similar resolution. As a result of this dramatic opposition, the Chan- 
cellor, on November 5, 2003, “paused implementation” of the June 17, 2003, 
resolution, and invited faculty to lead the development of a revised assessment 
proposal. Additional support for the faculty position was forthcoming. On 
November 12, 2003, the Association of Presidents of Public Community 
Colleges (APPCC) drafted a letter to SUNY Chancellor Robert King arguing 
that university-wide assessment would compromise rather than promote aca- 
demic quality and accountability. And, on November 19, 2003, the SUNY 
Community College Chief Academic Officers submitted their own letter to 
the Chancellor questioning the wisdom of adding one more layer of assess- 
ment to the several already in place. Objections in both of these letters focused 


281 



LITERARY STUDY, MEASUREMENT, AND THE SUBLIME: DISCIPLINARY ASSESSMENT 


on inevitable “teaching to the test” and the resulting narrowing and lack of com- 
patibility with the curriculum, plus the potential for politically injurious compar- 
isons of campuses whose missions and student bodies differ in significant ways. 

By the beginning of December, 28 of the 30 community colleges within the state 
university system had drafted their own resolutions in support of the FCCG reso- 
lution opposing the implementation of a plan for system-wide assessment. 

2004 

In March 2004, a system-wide committee (made up of faculty, students, 
presidents, chief academic officers, and System Administration staff members) 
met to discuss university-wide assessment, which had been newly named 
“Strengthened Campus-Based Assessment” (Board of Trustees, “Resolution 
on Implementation”). The system-wide committee developed a proposal to be 
presented to the FCCC and UFS. Under the proposal, system-wide assessment 
would be limited to three of the original sets of outcomes: mathematics, criti- 
cal thinking (reasoning), and basic communication-written, all of which were 
to be assessed by “nationally-normed” measures. The other nine outcomes 
could be assessed by strategies developed on individual campuses. Additionally, 
SUNY would pay for the “nationally-normed” measures such as the Academic 
Profile, (AP), produced by Educational Testing Service (ETS), and the Collegiate 
Assessment of Academic Proficiency (CAAP), produced by ACT. Thus, colleges and 
institutions would not have to bear the expense of these exams in order to 
meet SUNY mandates; however, we could only wonder: where was the real- 
ization that whatever money was spent essentially came from the same pot? 

Two other options were also included, but not funded, in the resolution. 
The two alternatives to the national standardized tests were categorized as 
“SUNY-normcd Measures.” These measures were offered to those “campuses 
wishing to include SUNY-normed measures in lieu of nationally-normed 
measures.” One of the following options could be selected: 

1 . A locally developed instrument that measures the learning out- 

comes in one or more of these three areas and that is demon- 
strated to correlate statistically (i.e., have concurrent validity ) with 
nationally-normed measures 

2. A locally developed instrument that measures the learning out- 
comes in one or more of these three areas that is reviewed and 
approved by the GEAR Group. As part of this process, GEAR 
will rely on discipline-based panels of distinguished SUNY facul- 
ty that will develop standards and rubrics campuses may use to 
assess student performance. Campuses choosing to use their own 
standards and rubrics must demonstrate to GEAR that their stan- 
dards and rubrics are essentially equivalent to those developed by 
the discipline-based panel. 

And finally, there would no longer have to be two levels of assessment. 

“Assessment for Improvement” and “Assessment for Accountability” could now be 
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conflated into one assessment process. That there were no “nationally-normcd 
measures” for writing and critical thinking that could be mapped to the SUNY 
learning outcomes seemed not to be a factor in the establishment of these options. 

It was the two proposed alternatives to “nationally-normed” measures 
that the FCGG focused on when they proposed a modified resolution on April 
3, 2004 (“Resolution on Implementation of Strengthened Campus-based 
Assessment”). Specifically, the FCGG resolution, if implemented, would 
require System Administration to provide funding for the two alternative 
assessment approaches commensurate to what it would have had to pay for 
the implementation of the national standardized tests. In addition, the FGGC 
response required that the discipline-based panels have equal representation 
from the community colleges and the state-operated institutions. The resolu- 
tion provided for the discipline-based panels to be selected by the SUNY gov- 
ernance bodies in collaboration with GEAR. And finally, the standards and 
rubrics developed by the discipline-based panels would have to be submitted 
to the governance bodies for their approval before they were used by GEAR 
to approve locally developed instruments. In other words, the FCGC agreed to 
the compromise if adequate funding was provided for campuses to resist stan- 
dardized measures and if they had significant input and control over the cre- 
ation and implementation of those assessment measures. The Faculty Senate 
endorsed the FCGG resolution that same month and the resolution was sent 
to the Chancellor. 

In June of 2004, the Board issued a new resolution adopting the compro- 
mises specified by the FCCG and UFS resolutions. A victory appeared to have 
been won, but it would only be a real victory if the two alternatives to the 
standardized testing were made viable. In order to create this viability, 

GEAR — in consultation with the FCGG and the UFS — established three 
discipline-based panels: one for critical thinking, one for mathematics, and 
one for basic communication-written. Each of these committees was charged 
to create an assessment instrument that could serve as the template for a state- 
normed assessment process. Members of these committees represented (again) 
all divisions of the state system. 

2005 

Once the three discipline-based panels had completed their work and 
System Administration had conferred with nationally recognized testing 
organizations, those in charge of assessment at the provostial level announced 
that they planned to organize a conference at which representatives of ETS 
and ACT would be invited to come and present the possibilities currently 
available through their organizations for conducting these assessments. Both 
groups also indicated that they would present alternatives they could devel- 
op that would satisfy the mandate now in effect. Through our positions as 
officers of the SUNY Council on Writing (Pat was President; Tina was 
Secretary), we protested the absence of any voice that could contribute the 
ideas and strategies for valid and reliable assessment of writing that had 
been developed by the discipline of composition and rhetoric over the 
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years. We were told that there was no money to pay for any presenters. 

ETS and ACT were, of course, coming free of charge. The SUNY Council 
on Writing (affectionately known as SUNYCOW) agreed to contribute 
money toward the honorarium of a composition scholar. We were gratified 
that Kathleen Yancey 7 was then invited and attended the conference. 

At the SUNY general education assessment conference, held in Syracuse 
on April 27-28, 2005, each of the discipline-wide committees presented the 
rubrics that it had developed and that campuses could use in lieu of standard- 
ized tests. The Writing Discipline Committee, composed as it was of quite a 
few members who already knew one another through SUNYCOW, had 
worked intensely, meeting once in person and then finishing its document via 
email. 8 When our results were presented to the group at the conference, there 
were almost no objections raised. The rubrics were praised by almost everyone 
present. (We have included these rubrics in an appendix.) When, however, the 
mathematics rubrics and the critical thinking rubrics 9 were presented by the 
respective committees in these areas, wide disagreement and objections were 
voiced by those attending. 10 We concluded that the near unanimous accept- 
ance of the writing rubric was basically a product of the fact that so many of 
us in the state came together at regular intervals at conferences of SUNY- 
COW and had established a conversation about assessment that spanned 
many years. Those of us on the committee who shaped the rubric had the 
advantage of all those conversations. 

In addition, representatives of both ACT and ETS presented sample 
questions that could become a part of new tests each group was proposing. 
The inadequacy of most of the sample items presented to assess the writing 
outcomes was immediately apparent to almost all attendees, even those not 
directly concerned with this particular general-education area. 11 From our 
point of view, we felt as though we had created a set of rubrics that most cam- 
puses would accept in lieu of a standardized test. And, indeed, a substantial 
majority of campuses used our rubrics. 

After 2006: Basic Communication Assessment 

SUNY defines college writing mainly as argument-centered. Note the 
SUNY outcomes for basic written communication: 

• Produce coherent texts within common college-level written 
forms; 

• Demonstrate the ability to revise and improve such texts; 

• Research a topic, develop an argument, and organize supporting 
details. 12 (Provost’s Advisory Task Force on General Education) 

And because the infused competency of critical thinking (reasoning) is also 
commonly being assessed through written communication, the following criti- 
cal thinking outcomes further emphasize the link between college writing and 
argument: 
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• Identify, analyze, and evaluate arguments as they occur in their 
own or other’s work; and 

• Develop well-reasoned arguments. (Provost’s Advisory Task Force 
on General Education) 

These outcomes, with the exception of the outcome relative to revision, do not 
reflect the developing composition theories of the past two decades, but rather 
resort to the forms prescribed by current-traditional rhetoric which have ruled 
the teaching of writing in American institutions for over a hundred years — the 
same forms that ETS and ACT have successfully argued that they can, in fact, 
measure with an acceptable degree of accuracy. 13 In order to provide a means 
of some resistance to these outcomes and to bring into the conversation nation- 
ally created documents, the Writing Discipline Committee prefaced its report 
with assessment and outcomes statements made by the Council of Writing 
Program Administrators 14 and the Conference on College Composition and 
Communication. It seems quite relevant to us that, if campuses are to be held 
to “national standards” for the assessment of writing, these should all be 
informed by policy statements developed and approved by national organiza- 
tions within the discipline of writing. As a result of the wide distribution of the 
Writing Discipline Committee report, we have been able to begin informing 
the academy of current composition theories and methodologies and why stan- 
dardized testing cannot be considered a valid measure of writing ability. 

While our Writing Discipline Committee was debating and collaboratively 
constructing rubrics, members of the GEAR committee began looking at 
available tests for assessing research writing. They were not able to locate any 
serviceable standardized research test that had been nationally normed. As a 
consequence, the research outcome and the rubric that the Writing Discipline 
Committee had crafted to assess it were removed from Strengthened Campus- 
Based Assessment. Campuses were now free to design their own assessment 
for the research outcome. 13 

What is perhaps the most significant element of the move toward 
strengthened campus-based assessment was that it brought scholars from a 
variety of colleges and universities together to discuss the teaching and learn- 
ing of writing and how those ideas might best be articulated to those outside 
the discipline. During the 2006-2007 academic year, several centrally located 
workshops were organized by the Provost’s office and GEAR. Campus response 
to this invitation was overwhelming, with more than 350 faculty and staff reg- 
istering to attend. At each one of these workshops, the workshop leaders were 
either GEAR members or members of one of the discipline-area committees 
(writing, mathematics, and critical thinking). Those of us conducting the work- 
shops on the writing rubric brought sample papers that we had, as a group, 
scored against our rubric and asked those present to go through the same 
process we had gone through: reading the papers for each set of the writing 
outcomes and scoring them accordingly. 16 We came away with the sense that 
some faculty at least were beginning to understand how they could “own” the 
rubrics and even tweak them a bit to serve the needs of their own campus. An 
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evaluation conducted by GEAR supported our initial reaction that these work- 
shops helped faculty see the value of their assessments as primarily a prod to 
greater interaction among themselves about all aspects of the teaching of writ- 
ing: assignments, commenting, peer groups, revision, and grading. 1 7 
Furthermore, that we used mostly the same papers at workshops throughout 
the state gave us some sense that there was a fair degree of commonality from 
campus to campus, but also a need for recognition of differences between and 
among campuses. Finally, as workshop leaders, we could not avoid the conclu- 
sion that every campus interpreted the rubrics in the context of its own pro- 
grams and student body. That is, one campus did not read a given rubric 
exactly the same way as another campus. Those who love standardization may 
not like this truth, but, as humanities teachers, we know that such differences 
are a part of all reading. And this is as it should be. Further studies of the results 
(not yet in preparation) might reveal the degrees of commonality and difference 
among campuses and could lead to further collaboration among schools as a way 
of sharing and learning from each other’s assessments. 

The Humanities Rubrics and the Major 

As members of English departments ourselves, we traced separately the his- 
tory of mandated assessment of the humanities and English studies. Assessments 
of both were mandated in the November 28, 2000 Provost’s Task Force on the 
Assessment of Student Teaming Outcomes, but neither of these areas was 
included in the strengthened campus-based assessment alterations of 2004 
(that is, no requirement that some nationally standardized measurement be 
used was mandated). The humanities area, however, was still to be assessed at 
the campus level with the standardized outcome established and mandated by 
the Provost’s Task Force on General Education for all campuses, which speci- 
fied that students are to 

demonstrate knowledge of the conventions and methods of at least 
one of the humanities in addition to those encompassed by other 
knowledge areas required by the General Education program. 

There is no outcome specifically directed at literature; literature, therefore, is 
subsumed under the larger division of the humanities. In other words, a liter- 
ature course is required for an undergraduate degree within SUNY only inso- 
far as it fulfills a humanities requirement. A student can achieve an under- 
graduate degree within the SUNY system without ever taking a literature 
course unless the local institution requires a literature course within its own 
general education curriculum. Unlike written communication, literary study 
has lost its status as a defining tenet of an undergraduate degree within 
SUNY. It lies beyond the purposes of this essay to reflect on the fate of litera- 
ture studies in higher education; much is being written on this topic. 18 What 
we saw in our perusal of assessment in our state is that literature no longer 
has its own slot at many of our campuses. It has been subsumed in a variety 
of ways. 
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Observing this omission in the general education curriculum and failing 
to locate any real suggestions as to what the outcomes for an English major 
might be, we thought it would be worthwhile to examine how the individual 
SUNY colleges and universities were approaching assessment of the English 
major. Recommendations included in the year 2000 “Report of the Provost’s 
Advisory Task Force for the Assessment of Student Learning Outcomes” rela- 
tive to assessment of the major specified that each campus assume responsibil- 
ity for assessment of major fields and insure that such plans include the input 
of “faculty, students, professional staff, and administrators.” Furthermore, such 
assessments should occur “every five to seven years” (25) (an obvious attempt 
to coordinate such assessment with the schedule of accreditation reviews). 
Campuses were to include in their final report to System Administration the 

• Delineation of the programmatic goals and objectives for the 
Major, with an emphasis on the learning outcomes students should 
demonstrate as they progress through the program to completion; 

• Description of the programmatic activities that are intended to 
accomplish these goals and objectives; 

• Identification of the assessment criteria to be utilized in deter- 
mining whether or not students are meeting the program’s goals 
and objections; and, 

• Description of the processes to be used in assessing the program, 
making changes, if suggested . . . and disseminating assessment 
results.... (23) 

We thought we would find patterns of commonality and dispute surrounding 
the assessment of student learning within the English major which we could 
then present at the Modern Language Association (MLA) convention in 
December 2002. We hoped to continue a discussion as to what we in the disci- 
pline of English value in student learning in the broader forum of the MLA 
convention, and in so doing, find ways to articulate and validate those values 
through the use of assessment methodologies. What we discovered was an 
extraordinary silence. 

Although the deadlines for assessment plan submissions had passed and, 
in fact, first-year reports were due, we noticed that there were very few assess- 
ment plans submitted for any major. Most institutions had simply submitted 
their intended cycle of assessment areas. There were no assessment plans for 
English majors beyond boiler-plate plans that could be applied to any major, 
nor were there any reported outcomes for the English major. 

The first assessment cycles’ outcomes reports due were for the 2001-2002 
school year. Within this first cycle, we looked to see which colleges would be 
reporting on the English major. None of the research university centers chose 
to report on English in the first cycle. Seven colleges did not have an English 
major to assess. Instead English was subsumed within humanities, liberal arts or 
general studies majors. In addition, because the state of New York does not 
allow for an associate’s degree in English at the community colleges, the major 
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that most often specifically included an English emphasis was the associate of 
arts degree in the liberal arts and sciences with a humanities or a humanities 
and social science emphasis. 

Thus, instead of discovering assessment methodologies that would help 
us articulate the value(s) of the English major, we discovered instead a 
devaluing of literary study within SUNY. This does not mean that individual 
colleges do not require literature. In fact, about half of the community col- 
leges do require at least one literature course for their liberal arts degree, 
whether it is an AA or an AS degree; however, the literature course is often 
cast as the second course within a composition sequence. Furthermore, 
assessments of a college education that do not include a separate category for 
literature or English studies inevitably contribute to the lessening of their rel- 
evance; this is always a result of “teaching to the test”: what the test does not 
test tends not to be taught. 

When we returned in June 2004 to examine the assessment plans and 
reports, we observed little change in the status of the files with regard to assessing 
the English major, although — as one might imagine from the discussion above — 
files were rife with writing assessment plans and reports. While it is likely that fac- 
ulty involved with the English major had not begun to substantively participate in 
articulating outcomes, and that assessing those outcomes and reporting on those 
assessments was due to a lack of funding and of consequences, it is clear that 
English faculty and administrators may be complicit in their own silencing within 
SUNY because of their resistance to assessment efforts. 

Although this chapter is not focused on the make-up of English depart- 
ments, we (as faculty who teach both literature and writing) have often found 
ourselves wondering if English departments are wise to allow writing to become 
its own program outside the construct of the English department — as has 
happened at many institutions. The general public tends to think of English 
departments as venues for the teaching of writing; this connection, in turn, 
strengthens the role of English departments in the minds of many. Our ques- 
tion, then, to those who are considering this divorce: are you weakening your 
department by allowing it? In turn, as we have seen by the written communi- 
cation outcomes stated above, writing is still being defined through current- 
traditional concepts despite the last forty years of composition theory and the 
last ten years of separating writing programs from English departments. Has 
this divide really elevated composition to a legitimate field of study in the 
academy or has it made it more vulnerable to the whims of administrators 
and faculty who have a very narrow view of what the teaching of writing 
should entail? 

Interesting Alterations 

From the outset, we expressed our fears that assessment results would 
become political and would, inevitably, be used to make comparisons between 
and among SUNY campuses. Our concerns met with little response whenever 
we voiced them. However, in fall 2004, Erika Rosenberg, a Gannett News 
Service reporter, requested an assessment presentation from SUNY. SUNY 
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prepared a notebook detailing the assessment process, GEAR, and diversity 
metrics. After the presentation, she requested the percentages of each out- 
come in each category (exceeding, meeting, approaching, not meeting) report- 
ed by each campus. The Chancellor denied the request stating that the data 
were not report cards. The reporter filed a request for information under 
FOIL (New York State’s Freedom of Information Law) requesting the data. 
SUNY was forced to comply and the campuses provided their individual 
assessment data within the context of each different assessment tool used. 

The response to the FOIL request was released at the SUNY Board of 
Trustees Academic Standards Committee meeting on January 10, 2005 with 
the trustees, faculty, and press present. At this point, it became clear to all 
involved that assessment reports could not be kept confidential. The inherent 
conflict between assessing for the improvement of student learning and assess- 
ing for the purpose of accountability became a reality. We do not wish to pur- 
sue other scenarios here but just to suggest, based on our experience, that 
those in charge at state levels ought to consider not the specific results from 
each campus, but instead, the specific plans each campus has for using its 
results to improve the learning of its students; that is a justifiable way to over- 
see a system. Of course, those of us on the GEAR group who were writing 
teachers with an understanding of audience long warned of potential misuse 
of data, but we were assured by system administrators that the reports could 
and would be kept confidential. When everyone had to acknowledge that this 
could not be the case, the entire SUNY assessment initiative was threatened. A 
Task Force on Assessment Reporting was formed, and in a memo dated July 
25, 2006 from Patricia L. Francis, Assistant Provost for University Assessment 
and Academic Initiatives, two important alterations were made to the prior 
mandates. 9 

First, the guidelines for Strengthened Campus-based Assessment had 
required that campuses submit a report that included the percentages of stu- 
dents “exceeding, meeting, approaching, and not meeting the delineated 
learning outcomes” (Provost’s Advisory Task Force on the Assessment of 
Student Learning Outcomes 16). The revised mandate eliminated that 
requirement, stating only that “campuses are still required to monitor the per- 
centages of students” in each of these categories; however, these percentages 
are not to be sent to a central file in Albany (Francis). There is no doubt in our 
minds that this particular change was a product of a growing concern about 
detrimental comparisons between and among campuses, a fear that — as we 
have already mentioned more than once — had existed for some of us from the 
very outset of the whole process. 

Second, original guidelines for assessment of the major had required that 
materials submitted include whatever reports had been composed by any exter- 
nal review team sent by accrediting/ certifying organizations, plus the self-study 
document prepared by the major department for this outside review team, and 
the “Summary Report Form for Assessment of Student Learning Outcomes in 
the Major” (Francis). The revised mandate eliminated the last two items. In 
effect, assessments of the major have been equated with reports of outside 
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accreditation organizations. And in effect, all the state efforts directed at assess- 
ing majors have resulted in no change at all, so far as majors are concerned. 

Advice 

On the basis of our experience with the whole process, we offer the fol- 
lowing advice and suggestions, mixed with precautions: 

1 . As the pressure grows at the national level for postsecondary assess- 
ment, it is imperative (in our minds, at least) that disciplinary organi- 
zations pre-empt the issue. National organizations such as the 
Modern Language Association, American Historical Association, 
American Philosophical Association, American Mathematics Society, 
and so forth, must create their own assessment committees primarily 
made up of classroom teachers. These committees need to design 
outcomes for their disciplines and develop assessment strategies. At 
present, there are already too many organizations focused on assess- 
ment as such, and these organizations are speaking out about what 
assessment at this level should be. 20 But where are the voices of those 
who teach and administer within disciplines? Their voices must be 
primary, not secondary. Once these groups have completed their task 
and received the imprimatur of their organizations, they will be pre- 
pared to speak up and offer their plans. They will also be available to 
confer with groups at the state level which are pressing for assessment. 

2. As soon as there appears on the horizon any possibility of state- 
mandated assessment, faculty must unite. 21 Voices that speak for 
statewide groups rather than for individual campuses can be the most 
effective. In New York State, we were fortunate to have two faculty 
governance bodies with considerable influence and considerable expe- 
rience dealing with SUNY administration. Consequently, their voices 
were heard. Subject area groups can also mount an effective voice as 
SUNYCOW did. Our group has a statewide conference every year; 
consequently, all of us know our counterparts at quite a few other 
campuses. Our prior conversations made our political activity much 
more effective and much easier to mount. Unfortunately, many disci- 
plines do not have statewide groups that are able to react to local 
issues concerning assessment. Perhaps faculty governance groups and 
statewide discipline groups, wherever they do exist, should consider 
opening up more lines of communication with each other. Further- 
more, if the advice we offer in (1) above were to be followed, national 
disciplinary assessment committees would be available for consulta- 
tion and advice. 

3. Once a state’s governing body (whatever entity may have control over 
legislation involving higher education) begins to propose actual plans 
for initiating statewide assessment, concerned faculty groups usually 
have options: to sit on their hands or to participate (or insist on being 
included) in any forum that is scheduled by this body before it makes 
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any final decisions. Both of these possibilities come with dangers. If 
faculty participate in a scheduled forum and demonstrate intractabili- 
ty, the governing body may become even more convinced than it 
already is that faculty do not want the public to have any knowledge 
about whether their teaching is successful. Consequently, anyone who 
opts to speak at such a forum must be a consummate politician and 
study how to walk a very fine line between demonstrating a willing- 
ness to listen and an insistence upon expressing a perhaps contrary 
point of view. If faculty elect not to participate in a scheduled forum, 
they will need to be prepared to deal with undesirable legislation that 
they can, if they so choose, continue to ignore. However, resisting 
campuses, faculty, and students may face dire consequences in terms 
of accreditation, funding, transferability of credits, and negative press. 
The public today appears to have a love-hate relationship with higher 
education faculty. Parents want their children to have a college educa- 
tion, but are often negative about teaching successes and failures. 
Faculty can and should find ways to present a positive attitude toward 
assessment. The MLA itself can be helpful provided that whoever 
speaks for it shows an awareness of and a willingness to recognize the 
importance of well-conducted assessments, particularly at the campus 
level. Quite frankly, the indignant voices of individual faculty mem- 
bers often produce negative reactions. One of our purposes in this 
article is to present information that might lead faculty to be informed 
critics of any assessment plans presented to them by governing 
boards, deans, provosts, and presidents. Instead of indignant faculty 
voices, we need informed and critical voices. 

4. The above brings us to a further suggestion. Postsecondary faculty 
desiring to resist, participate in, reconfigure, concede (even if reluc- 
tantly), or to voice their concerns about assessment need to join with 
K-12 groups in their state. The National Council of Teachers of 
English (NCTE) has local affiliates in all states, and NOTE and these 
local groups have a long history of reaction to testing mandates. 
Local-area MLA groups could also be called upon. State boards and 
legislatures often do not respond well to out-of-state contributions; 
however, the more localized groups can be helpful. State-level groups 
can consult with their national organizations for guidance, however, 
before making public statements. 

5. Matters of funding need to be embedded in every single point of our 
advice. Those who negotiate with mandate-creating groups need to 
insist adamantly on adequate funding, constantly reminding legisla- 
tures and the public that faculty members have an obligation to assess 
students in their own classes, but no contractual obligation to assess 
the programs of which they are a part. Such work traditionally lies 
outside the responsibility of faculty. Since, however, faculty must be 
involved in designing outcomes and assessment, there needs to be 
extra compensation for this work that goes beyond their contractual 
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duties. Furthermore, legislative bodies and the public need to be con- 
stantly reminded that, if the information requested is so vital to deci- 
sion-making and curricular change, it cannot be gathered by 
untrained personnel. Institutions and faculty need guidance and time; 
otherwise the enterprise turns into busy work done to fulfill the man- 
date only nominally. Assessment can be important and meaningful; it 
cannot be done cheaply. 

To sum up the above five points: what is crucially important here is that 
groups of faculty within a given discipline come together to plan a response 
before governing bodies have a chance to construct plans that they then may 
become wedded to, plans that are not backed up by allocated funding. For 
humanities organizations such as the MLA, what is even more critical is that our 
voices be heard before some template of outcomes categories is constructed. 
Such early intervention may ward off the merging of literature into some large 
humanities category. We did not forestall that merger in New York State. 

6. Once legislation has been passed and is in force, we recommend con- 
tinued monitoring of it and continuous attempts to make changes in 
undesirable elements of the legislation. We were able to do that in 
New York State. Opponents to the legislation should not return to 
their individual corners, but should continue to press for change and 
continue to demonstrate that certain aspects of the mandated assess- 
ment are unworkable or inadequate. The statewide conference in 
which we participated and the subsequent workshops demonstrated to 
faculty that they can find ways to make mandated assessment suit 
their purposes. 

7. Obviously, one of the most important issues to press continuously at 
all levels of decision-making is the composition of all committees cre- 
ated: each such committee must be more than 50% faculty. 

8. For the sake of emphasis and because this chapter is mainly concerned 
with MLA matters, we repeat our recommendation in (1) above: the 
MLA must establish a panel of experts on assessment, particularly 
statewide assessment. Members of this panel could testify at statewide 
hearings, but more importantly, they could meet with concerned facul- 
ty on a state-by-state basis and offer advice, tactics, published statistics, 
and so forth. This panel needs to be composed primarily of classroom 
teachers who had had direct experience with various forms of man- 
dated assessment. Such a committee could: 

• help communicate what we recognize as the biggest issue for a 
majority of postsecondary faculty in this country: resistance to 
what they believe-and rightly so pcrhaps-cannot be measured, 
namely, their effect on a given student in one class for one semes- 
ter in comparison to another faculty person’s effect on a different 
student in a different class at a different campus in one semester. 


292 



A CAUTIONARY TALE ABOUT SYSTEM-WIDE ASSESSMENT IN THE STATE UNIVERSITY OF NEW YORK 


Any discussion of reasons for this statement would require anoth- 
er article. 

• help faculty better communicate ongoing discussions about 
grading: a high percentage of faculty state that they are already 
assessing students by awarding grades. 22 The state’s and public’s 
reply to this argument is that no one seems to know what grades 
mean, particularly when grading across classes and institutions 
seems so variable. Anyone who has participated in grading ses- 
sions knows that this argument is not without validity. Equally, 
anyone who has participated in grading sessions knows how valu- 
able they can be as a guide or corrective to one’s own standards. 
None of this means that there is no agreement and an MLA 
committee of assessment experts could help us all understand this 
and explain it to outsiders. We know the many good reasons why 
mandated assessment comparing campuses and faculty is simply 
not possible. 

• help us communicate our main argument: that assessment 
must be owned by faculty to be effective; for that to come about, 
faculty must be included in constructing the assessment so that it 
will be relevant to their own students and programs, which they 
know better than outsiders. Comparisons can only be valuable to 
a certain degree, for each campus has its own particular mission 
and its own student body. On the other hand, resistance to stan- 
dardized tests and standardized assessments may well not work in 
today’s climate. 23 

Perhaps this resistance continues because of the ever-present linkage 
between assessment and accountability, an awareness that assessment for 
improvement of student learning may not be the primary motivation behind 
national, state, and college assessment efforts. In fact, even the 2000-year 
report of the SUNY Provost’s Advisory Task Force on the Assessment of 
Student Learning Outcomes did not make clear what motivated its creation. 
What could have been the reason for demanding such an extensive reconsider- 
ation of the country’s largest higher education system? On the state level, 
some changes of note included the election of Republican Governor George 
Pataki in 1995. In addition, there were changes in the leadership of the state 
legislature. Of course, with the change in governorship came changes in 
SUNY’s Board of Trustees. Also by the end of the 1 990s, SUNY had a new 
chancellor and a new provost. This rather remarkable recasting of the admin- 
istrators within New York and SUNY inevitably led to changes in economic 
policies and higher education philosophies. 

Perhaps the most significant marker of these new economic policies and 
philosophies toward higher education is the document entitled “Rethinking 
SUNY” from which we quoted in the first paragraph of this article. The 
desire to provide high-quality education for all without wasteful expenses is the 
tension underlying all pedagogical, practical, and administrative decisions in 


293 



LITERARY STUDY, MEASUREMENT, AND THE SUBLIME: DISCIPLINARY ASSESSMENT 


American higher education, and certainly this is evident within the SUNY sys- 
tem. A decade after the 1995 proclamation, the SUNY Board of Trustees 
moved beyond its mandates in an attempt to develop a system to determine 
the degree to which those mandates were succeeding. That goal, in turn, led 
to the succession of events and regulations detailed in this chapter. 

To the American capitalist and democratic sensibility, which accepts com- 
promise as inevitable, the struggle to provide high quality for less money must 
seem not only laudable, but commonsensical. As Gregg Primo Ventcllo noted 
in a recent article, “The official documentation of assessment is an attempt to 
quantify our work in a profit-oriented capitalist culture that knows no other 
way to measure success than by counting it” (59). We in the humanities need 
constantly to make the argument to the public that our work cannot be quan- 
tified nor can it be reduced to some common denominator. But we need to 
make this argument with supporting details. Too often those who make this 
statement to public groups do it without supporting details. One of the tasks 
of an MLA committee might be to archive specific examples of classroom 
assignments and projects that complicate assessment. 

The “Report to the Teagle Foundation on the Undergraduate Major in 
Language and Literature,” produced by the MLA in 2009, notes rightly that 
“without literature, there is no in-depth understanding of narratives that lead 
to the discovery of other cultures in their specificities and diversity and to the 
understanding of other human beings in their similarities and differences” 

( 2 ). 25 us, this resonates with older validations of literature as a legitimate 
area of postsecondary education, validations that, within the discipline itself, 
have often been belittled. It is instructive that the discipline has returned to 
this earlier rationale after a period of high theory that often alienated out- 
siders through its seemingly abstruse and arcane vocabulary that, unfortunate- 
ly, was usually not translated into language the public could understand. Had 
it been, the public might well have seen the value of the theories. Those theo- 
ries, whether feminist, queer, postmodern, postcolonial, or deconstructionist, 
have enriched our classrooms in ways the public could understand if the lan- 
guage had been made more accessible. These theories, plus traditional close 
reading of texts, are the heart of any literature course. The humanities as a 
unit, say Pontuso and Thornton, do not exist 

merely to impart a body of knowledge, but to convey a way of 
thinking. Liberally educated students must be taught to analyze 
problems, evaluate data, critically appraise arguments and beliefs 
and, more importantly, weigh alternatives. In a sense the object of 
the liberal arts is to prepare young people to cope with problems 
and challenges that do not yet exist. (62) 

The difficulties of assessing courses designed to accomplish this list of 
objectives are not specific to English departments. Other fields including the 
sciences have comparable, and equally difficult to assess, goals. But another 
complicating factor touches on the students themselves. All of us rightly teach 
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the students assigned to our classes, and we know that they arrive there from 
quite diverse backgrounds with quite diverse prior educational successes and 
failures. Our goal is to help each student move from where s/he is to a higher 
plane; that higher plane may be for some exactly where some of their class- 
mates were at the beginning of the term. Nor can standardized assessments 
take into account the amount of effort students devote to a particular class, 
nor can it take into account the varied lives of our students: many of them 
have families, jobs, and outside commitments that eat into the time they would 
like to devote to our classes. Many of us have had students who simply wanted 
their “C,” students who recognized that their commitments permitted them 
only limited time for our subject. All of these factors need to be understood by 
the public and, for the most part, they are not. We need venues and strategies 
for getting this message out beyond the confines of academia. And, finally, 
standardized assessment has the unfortunate consequence of setting a stan- 
dard that is far too low for some of our brightest students. 

One final caveat. Both of us believe that assessment is not going away. We 
can barricade the walls, but eventually that will not be productive. One of the 
recommendations made in the “Report to the Teagle Foundation” stated that 
a “constitutional” element of baccalaureate degree programs in English 
should be “empirical research to assess the successes and shortcomings of the 
program” (3). The report elaborates on this recommendation by recognizing 
that the “results of program changes need to be documented and evaluated 
empirically, through the adoption of outcome measurements” (3). We would 
like to know what the committee meant by “empirical research.” The purpose 
of this committee was not to confront assessment, but it is instructive to note 
that “outcome measurements” crop up in the prose; we can only hope that 
there is follow-up on this issue. 

A second concern arises when reading the “Report to the Teagle Foundation.” 
In its introduction, the report notes that those who comprised the working 
committee for the report included “college presidents and deans, as well as 
distinguished members of the legal and medical professions and visiting con- 
sultants” (Executive Summary). We noticed immediately that this list did not 
include any member, or at least did not choose to label any member, as a 
classroom teacher. Perhaps this accounts for the report’s almost total disregard 
for pedagogy. Granted this was not the focus of the group; nonetheless, any 
discussion of the value of a liberal arts education in today’s world ought to 
refer to teaching in some way. We cannot emphasize strongly enough the con- 
nection between assessment and teaching and learning. 

Observations 

In our minds, there is no question that the once-intensive drive for assess- 
ment has been lagging in the past year. Why? Perhaps because of a change 
from a Republican majority in legislative bodies to a Democratic majority, 
though we would not attribute greater understanding of assessment issues to 
the latter. Perhaps from waning interest on the part of a new provost. Promised 
results have not been evident: transfer for students has not been easier, confi- 
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dentiality has been compromised, and none of the promised feedback to the 
submitted reports has come about, nor have visits by SUNY system adminis- 
trators to individual campuses. The economic downturn is undoubtedly rele- 
vant. Funding has lagged far behind actual completion of assessment reports. 
For example, at SUNY Stony Brook, participants in the writing evaluation 
cycle received the promised compensation eight months after the evaluation 
was completed and only after continued nagging. At the time of this writing, 
the latest SUNY budget request includes only $300,000 for assessment. That’s 
hardly enough. Perhaps initiatives, such as this assessment initiative, always 
have a natural life span, and organizations move to other projects, leaving for- 
mer ones to drift and assume secondary importance. 

Failures of this nature have led faculty to rethink the whole assessment 
project and suggest that the assessment procedures mandated by accreditation 
authorities — in our case, the Middle States Commission on Higher Education — 
are sufficiently rigorous. As a result of this resolution, increased accreditation 
standards for assessment, and financial pressures, a Provost’s Advisory Group 
on the SUNY Assessment Initiative was convened and recommendations for 
streamlining assessment were issued in their September 2009 Report. These 
recommendations essentially undercut the Strengthened Campus-based 
Assessment mandate. This faculty resolution was followed up by a resolution 
on the part of the SUNY Board of Trustees essentially undercutting almost all 
prior assessment activity as detailed in this article. 

The authors of this piece continue to believe that faculty (or, at least, writ- 
ing faculty) learned much during this whole process. Could the money expended 
have been spent more effectively? We believe so, but that would have required 
the SUNY Board of Trustees to listen to us at the outset. One thing is certain, 
however: assessment will always be entangled with politics and economics. 

Those who continue to assert that nationally standardized testing can tell 
us how well thousands of colleges and universities are performing their func- 
tion are living some kind of myth. Can assessment at the campus and depart- 
ment level help us understand how well our programs are functioning? Yes. 
Can we learn something by comparing our results and assessment methods 
with other postsecondary institutions with similar goals and student bodies? 
Absolutely. But these assessments need to be designed, administered, and eval- 
uated by those who are directly involved in teaching the skills being tested. 
Much is learned by establishing outcomes, creating methods for assessing 
those outcomes, and collaboratively scoring the results of any assessment. We 
believe that the methods for assessing writing in New York State achieved 
what we considered our first goal: working together as teachers to improve 
what happens in all of our classrooms. In whatever guise assessment crops up, 
nationally or locally, those of us who care about students need always to link 
assessment to pedagogy, that is, to improvement of the teaching and learning 
environment in the classroom. The link between accountability and assess- 
ment will always, for better or worse, be headlined by others. It is up to us to 
find ways to “own” outcomes and assessment and to stress always, in every 
venue, the interactions between collaborative assessment and improved learning. 
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Postscript 

Since we completed this article, the assessment mandate for SUNY has 
almost completely reversed itself. In March 2010, the Board of Trustees 
rescinded the resolution that had established Strengthened Campus-based 
Assessment in June 2004 while claiming that SUNY “and its campuses have 
been at the forefront of assessment and quality assurance for over three 
decades, and have been a national model of best practice” (Zimpher). The 
major reason, and a valid one, given in this Resolution for the rescinding is 
that the “Middle States Commission on Higher Education and programmatic 
accrediting bodies have significantly increased in the rigor of their assessment 
standards in recent years in response to federal policy” (Zimpher). It is inter- 
esting to note here that the standards published by Middle States include the 
statement that “[wjhile the Commission expects institutions to assess student 
learning, it does not prescribe a specific approach or methodology” (qtd. in 
Zimpher). Further, the document recognizes that the requirement for assess- 
ment of the major essentially duplicated the work necessary for satisfaction of 
Middle States Accreditation. Consequently, that process now satisfies the 
SUNY requirement also. The resolution, in keeping with the Middle States 
policy as quoted above, also “permits each campus to develop assessment 
plans that are consistent with its mission and goals” (Zimpher). We, of course, 
argued for exactly this over ten years ago. The document claims that the 
Provost will consult with representative groups in order to create “an assess- 
ment advisory group,” presumably to serve as a resource for SUNY assess- 
ment efforts of all kinds. To our knowledge, that has not yet occurred. And, 
finally, the document makes no change to the General Education outcomes 
that are mandated to set the parameters for individual campus assessments. 

We surmise that the current economic recession that has hit SUNY hard, 
the appointment of a new Chancellor, a new interim Provost, a change in the 
composition of the Board of Trustees, and a new Chairman of that Board 
may well have engendered these changes. It is, of course, difficult to know 
what weight each of these changes carried. However, there is now an environ- 
ment of growing respect for faculty and faculty governance in matters of cur- 
riculum and assessment; this encouraging change is demonstrated through 
increased collaboration between faculty and SUNY administration. 

We would like to believe, and know we have the grounds to believe, that all 
of the work of all these committees at the state and individual campus levels was 
not a waste. All of us learned much by participating in these discussions, argu- 
ing, agreeing, disagreeing, coming to a degree of agreement anyway, and simply 
having the time to talk about assessment. Such talk has still been minimal on 
campuses. We can only hope it continues on individual campuses and between 
and among campuses; the groundwork for such interactions has been laid. 

We hope too that our narrative will be useful to those in other states who 
may face or who are already facing the possibility of postsecondary statewide 
assessment. We believe that much can be learned from our experience, both 
negative and positive. But most of all, we hope our narrative will lead to con- 
tinued thoughtful discussions of assessment and its role in education. 
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For responding to our continued need for information and updating, we would like to extend 
particular thanks to Patricia Francis, who was the Assistant Provost for Assessment during 
most of these years, Kimberley Reiser, President of the Faculty Council of Community Colleges, 
and to Sijffolk Community College’s faculty governance for archiving relevant documents. 


NOTES 

1 The Charge to State University of New York (Chapter 82, Laws of 1995), 
§ 135. Notwithstanding any provision of law to the contrary, the board of 
trustees of the state university of New York shall develop a multi-year, 
comprehensive, system-wide plan to increase cost-efficiency in the contin- 
uing pursuit of the highest quality and broadest possible access consistent 
with the state university mission (qtd. in “Rethinking SUNY” 1 2). 

~ The entire text of the Board’s resolution can be found in Appendix A of 
the SUNY Provost Advisory Task Force on General Education’s docu- 
ment, “Implementation Guidelines for SUNY Baccalaureate Candidate 
General Education Requirement.” 

0 The “knowledge and skill” areas are mathematics, natural sciences, social 
sciences, American history, western civilization, other world civilizations, 
humanities, the arts, foreign language, and basic communication. The 
“competency” areas are critical thinking (reasoning) and information 
management. Outcomes were created within each of the areas. 

^ For a timeline of events, see Reiser, “University-wide Assessment Chronology.” 

^ The document outlining the responsibilities of the GEAR committee and 
the principles by which they approved or disapproved the assessment 
plans of individual campuses is available at <http://www.cortland.edu/ 
GEAR/ Gearprocess.html>. 

® The community colleges are within the SUNY system and receive state 
funding, but they have an additional county sponsorship as well as their 
own respective boards of trustees. They, therefore, operate under different 
funding procedures and policies than the state-operated campuses. 

^ Kathleen Blake Yancey is Kellogg W. Hunt Professor of English and 
Director of the graduate program in Rhetoric and Composition at 
Florida State University. She is a Past President of both the National 
Council of Teachers of English and the Conference on College 
Composition and Communication. Currently, she is the editor of College 
Composition and Communication, the leading journal in the field. 

The following were members of the Writing Discipline Committee: 

Chair: Pat Belanoff, Professor, English, Stony Brook University; Mili 
Clark, Director of Composition, SUNY Buffalo; Aniko V Constantine, 
Distinguished Teaching Professor, English/Humanities, Alfred State 
College; Wayne Fulks, Assistant Professor, Liberal Arts, Sullivan County 
Community College; Jake Holden, Assistant Professor, English, Fulton- 
Montgomery Community College; P. Kathleen McCoy, Associate 
Professor of English, Adirondack Community College; Robert Moore, 
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Professor, English, SUNY Oswego; Maria Palmara, Assistant Professor 
and Chair, English, Modern Languages, and English as a Second 
Language, Hudson Valley Community College. 

^ For a spirited discussion of “critical thinking” as an entity to be subjected 
to standardized testing, see Griffin, “The Assessment Impasse.” 

I® A summary account of the SUNY General Education conference is avail- 
able at <http://www.cortland.edu/gear/gearconf2005.html>. 

1 1 For example, one of the representatives posited the type of question his 
company might develop to assess revision. The example began with a short 
paragraph of four or five sentences. The instructions stated clearly that the 
paragraph was accurate as it stood. However, a part of one of the sen- 
tences was underlined. The multiple choices following the short paragraph 
presented alternate ways of completing the partially underlined sentence. 
The student was then to pick the answer that best fit in with the suggested 
change. The right answer was not to change the meaning of the original 
sentence and had to be syntactically and grammatically correct. Such an 
example, of course, makes a mockery of revision: the test maker revises 
half of the sentence and the student finishes that task. And, of course, revi- 
sion always must begin with consideration of an entire written piece. 

This outcome, “Research a topic, develop an argument, and organize sup- 
porting details” remains one of the outcomes to be assessed for written 
communication, but it is no longer a part of “Strengthened Campus- 
Based Assessment,” that is, there is no longer a requirement that some 
form of “national standard” be applied. 

1 ^ For a more thorough discussion of “current-traditional rhetoric,” see 
James A. Berlin, “Current- Traditional Rhetoric: Paradigm and Practice.” 
^ The Outcomes Statement can be found in Council of Writing Program 
Administrators’ “WPA Outcomes Statement for First- Year Composition.” 

1 ^ For the record, and with the hope that others might find them useful, we have 
included the research rubric in the Addendum with the other two rubrics 
' 6 See SUNY GEAR, “Report of the Writing-Discipline Committee” for 
these rubrics in their entirety. 

1 ^ Electronic surveys conducted after the workshops revealed that partici- 
pants generally viewed the workshops in a highly positive fashion. As just 
a few examples of the survey results: 

• 78% of respondents indicated they were very satisfied or satisfied 
with the facilitator-led sessions with respect to their ability to use 
the rubrics. 

• 71% of respondents indicated they were very satisfied or satisfied 
with the facilitator-led sessions with respect to their ability to 
teach others on their campus to use the rubrics. 

• 86% of respondents indicated they were very satisfied or satisfied 
with the overall quality of the facilitator-led sessions. 

• 86% of respondents indicated they were very satisfied or satisfied 
with the overall quality of the workshop. 
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William Chace notes that “while the study of English has become less 
popular among undergraduates, the study of business has risen to become 

the most popular major in the nation’s colleges and universities Here is 

how the numbers have changed from 1970/71 to 2003/04 (the last aca- 
demic year with available figures): 

English: from 7.6 percent of the majors to 3.9 percent 
Foreign languages and literatures: from 2.5 percent to 1/3 percent 
Philosophy and religious studies: from 0.9 percent to 0.7 percent 
History: from 18.5 percent to 10.7 percent 
Business: from 13.7 percent to 21.9 percent.” 

1 8 See Francis for a link to the memo, and “Assessment Reporting Task Force 
Recommendations” for an additional clarifying list. This last list, dated 
July 2006, was undoubtedly a product of the confusing back-and-forth 
between campuses and system administration because of changes and 
misunderstandings. It clarifies the mandates in effect at that time. 

90 & 

- u For information on the formation of new assessment-focused groups, see 
Jaschik, “Assessing Assessment.” 

Since completing this article, we have learned of a project undertaken by 
the Lumina Foundation for Education that seeks to create some degree of 
consistency in the granting of degrees in fields such as history, chemistry, 
and physics. Three states are participating in the project and will be work- 
ing toward establishing outcomes within these major fields. The question 
they will be addressing is, as an example, “What should a graduate with a 
degree in history know and be able to do?” It appears that Lumina is not 
intent on enforcing outcomes, only in getting those in the same fields at 
different institutions to share and come up with guidelines. For more on 
the project, see Jaschik, “‘Tuning’ College Degrees.” 

22 There seems to be a good deal of confusion among faculty, administra- 
tors, and the public between what it means to “assess” and what it means 
to “evaluate.” To see a discussion of how these two terms have evolved to 
indicate different processes within the field of outcomes assessment, see 
“Power and Agenda Setting in Writing Assessment” in Assessment of 
Writing: Politics, Policies, Practices by Edward White. 

^3 The strength of the opposing sides in this debate, to participate or ignore 
government calls for assessment, shows up in the comments to Scott 
Jaschik’s article in Inside Higher Ed, “Assessing Assessment.” 

24 For an interesting discussion about how assessment threatens what faculty 
value in the classroom see Ventello, “The Assessment Edict and the Love 
of Teaching.” 

23 In 2006, the MLA responded to the Teagle Foundation’s invitation to 
think about “the relationship between the goals and objectives of under- 
graduate concentrations in their disciplines and those of a liberal educa- 
tion” (qtd. in MLA 1). 
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APPENDIX A: BASIC COMMUNICATION OUTCOMES SUNY 

Students will demonstrate their abilities to produce coherent texts 
within common college level forms 

Exceeding: 

Writer presents an easily identifiable, focused, original, and thought 
provoking controlling purpose or thesis. The paper moves coherently, 
logically, and even creatively from an engaging introduction to a well- 
demonstrated conclusion. Paragraphs fit within this structure coherent- 
ly and present pertinent examples and evidence to support central and 
subsidiary ideas. Sentence structure displays sophistication and variety; 
transitions add to the logical development of the topic. The essay 
exhibits a solid command of word variety and a tone and diction 
appropriate for the subject and its implied audience. Mechanics (gram- 
mar, punctuation, spelling and documentation, if needed) are nearly 
flawless. 

Meeting: 

Writer presents an identifiable and focused controlling purpose or thesis. 
The paper moves coherently and logically from a satisfying introduction 
to a solid conclusion. Paragraphs fit within this structure and present 
examples and evidence to support the ideas presented. For the most 
part, sentences are well constructed and transitions are sound — though 
the sequence of ideas may occasionally be awkward. The essay exhibits 
some degree of control over the tone and diction appropriate for the 
subject and its implied audience. Mechanics (grammar, punctuation, 
spelling and documentation, if needed) are mostly accurate, and para- 
graph transitions are sound, but the sequence of ideas may occasionally 
be awkward. 

A pproaching: 

Writer presents a wandering, vague, or unfocused controlling purpose or 
thesis. The paper moves awkwardly from a weak introduction to a con- 
clusion that does not adequately represent the body of the paper. Basic 
paragraphing exists, but often fails to support or even recognize a central 
idea, and the use of evidence and examples is inadequate. Sentence and 
paragraph transitions are often unclear, awkward, indirect, and/ or illogi- 
cal. Tone and diction are often inconsistent and/ or inappropriate for the 
subject and its implied audience. Mechanics (grammar, punctuation, 
spelling and documentation, if needed) are not well executed and may, at 
times, obscure meaning. 
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Not Meeting: 

Writer fails to present a controlling purpose or thesis; consequently it 
is difficult to identify exactly what the thesis is. The essay moves from 
an unsatisfactory introductory paragraph to an ending that does not 
serve as a conclusion, thus conveying the sense that much of what has 
been presented is unresolved. Sentence structure is often awkward 
and transitions are ineffectual and/ or abrupt or simply missing. 
Diction, tone, and word choice are not appropriate for the subject or 
for the implied audience. Mechanics (grammar, punctuation, spelling 
and documentation, if needed) disrupt reading and often obscure 
meaning. 

Students will demonstrate the ability to revise and improve such 
texts. 

Exceeding: 

Writer demonstrates clear evidence of an ability to revise by altering con- 
tent and approach, by reorganizing material, or by clarifying and 
strengthening the coherence of ideas. Alterations may include the addi- 
tion of new material, the deletion of unhelpful material, the substitution 
of more relevant material for less relevant material, the strengthening of 
transitions, introductions, and conclusions, and the rewriting of individual 
sentences. The mechanics (grammar, punctuation, spelling and documen- 
tation, if needed) of the final revision are nearly flawless 

Meeting: 

Writer demonstrates the ability to revise by refining the content, sharpen- 
ing the focus, and improving structure, clarity, and coherence. Refining 
content may include clearer presentation of evidence, shifting of emphasis 
to foreground the most relevant material, providing improved transitions 
that keep the focus evident, and reworking the introduction or conclusion 
as well as rewriting individual sentences. The mechanics (grammar, punc- 
tuation, spelling and documentation, if needed) are mostly accurate and 
rarely impede meaning. 

A pproaching: 

Writer demonstrates a lack of ability to revise in any substantial way. 
Whatever revision has been done has not been sufficient to improve the 
content, focus, structure, clarity, and coherence of an earlier draft. Such 
revision may very well be limited to sections of the essay and demonstrate 
a lack of awareness of how even small changes can affect the entire paper. 
Mechanics (grammar, punctuation, spelling and documentation, if need- 
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ed) have either not improved significantly or appear to be the only focus 
of the revision. 

Not Meeting: 

Writer demonstrates a lack of ability to revise at the level of content of 
structure. Either changes do not improve these features or are focused 
almost solely on mechanics. 

Students will research a topic, develop an argument, and organize 
supporting details. 

Exceeding: 

Writer indicates /presents a clearly evident, original, and sophisticated 
controlling purpose, responding creatively to the assignment prompt with 
evidence drawn from carefully selected sources, documented in accepted 
style. Conclusions are based on thoughtful integration of the students’ 
own thinking and careful analysis of the outside sources. Mechanics 
(grammar, punctuation, and spelling) are nearly flawless. 

Meeting: 

Writer indicates /presents a clear controlling purpose, responding intelli- 
gently to the assignment prompt with evidence drawn from appropriately 
selected sources, documented in accepted style. Conclusions demonstrate 
the writer’s conscious attempts to integrate his or her own thinking with 
an analysis of outside sources. Mechanics (grammar, punctuation, and 
spelling) are mostly accurate and rarely impede meaning. 

A pproaching: 

Writer indicates /presents either a shifting or unclearly articulated pur- 
pose, perhaps failing to focus on the assignment prompt. Outside sources 
may be inappropriate to the topic, or information from sources may be 
presented without careful analysis, and it may be inadequately document- 
ed. The conclusions may demonstrate little evidence of the students’ own 
thinking, presenting mainly a summary of the sources. Mechanics (gram- 
mar, punctuation, and spelling) are not well executed and may, at times, 
obscure meaning. 

Not Meeting: 

Writer indicates /presents little sense of a controlling purpose, failing to 
respond to the assignment prompt. There may be inadequate reference to 
outside sources, selected sources may show little apparent connection to 
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the assignment, or paraphrases may be unclear, quoted material may 
seem not to relate to the topic, and/ or there may be significant problems 
with documentation. The paper may consist largely of quotations and 
paraphrases from sources with few connections between and among them. 
The conclusions may demonstrate no evidence of the students’ own 
responses to the outside sources and may merely restate some of the ideas 
presented. Mechanics (grammar, punctuation, and spelling) disrupt read- 
ing and often obscure meaning. 
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READING FOREIGN LITERATURE CRITICALLY: 
DEFINITIONS AND ASSESSMENT 

JENNY BERGERON AND RUSSELL A. BERMAN 


HEIGHTENED SCRUTINY OF HIGHER EDUCATION has become a 
fact of our national life. Both the education community and the public at 
large are raising questions regarding the fundamental rationale for higher 
education, the specification of learning goals, the evaluation of outcomes 
and, of course, the constant increase of costs. Higher education is being 
asked to justify itself and to reform, where appropriate. No one should 
imagine that this challenge will go away. On the contrary, the interrogation 
is likely only to grow sharper, and the need for college and university faculty 
to examine their teaching practices and their underlying assumptions will 
continue unabated. Yet rather than adopting a defensive posture or other- 
wise resisting these questions, higher education can face these challenges as 
an opportunity to rethink and to clarify learning goals and pedagogical 
strategies. 

College learning in the United States has long been defined in terms of a 
mixture of general education and specialization in a major. Earlier debates 
frequently concerned the choice of subject matter, i.e., which texts should be 
taught in required general education or the extent to which majors should or 
could include prcprofessional — rather than purely academic — learning. In 
today’s context, however, the discussion has shifted away from specific materi- 
al, i.e., the texts to be taught, to a specification of the skills or capacities which 
a college education should elicit in students. How can college curricula enhance 
student cognitive growth? And can we measure that growth in particular abili- 
ties in ways that can demonstrate the effectiveness of education? 

In fact, the expectation that college students grow in intellectual maturity 
is hardly new, nor is the assumption that such maturity ought to be evidenced 
in certain abilities, notably writing, but also reading and, more broadly, in 
capacities for argument, judgment, and interpretation. Complaints that college 
students perform insufficiently in these terms are as old as college life. This is, 
however, not grounds to dismiss these concerns. 1 On the contrary, shifts in dis- 
ciplinary self-understanding, research in teaching and learning, and more 
sophisticated understandings of assessment provide opportunities to rethink 
the alignment of pedagogy and goals. 
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This essay attempts to clarify learning goals with regard to a core compo- 
nent of college education — reading — specifically in relationship to foreign lan- 
guage and literature. We assume that reading literature in the original foreign 
language can contribute to student skills in reading in general, in cross-cultural 
hermeneutics, and in critical reading in particular. By critical reading we 
mean a higher-level engagement with the text that goes beyond effective 
understanding of the foreign language text. That is, it is more than literal 
translation, and it also surpasses a capacity to summarize or recount plot. 
Critical reading assumes those lower-level skills but also involves a capacity to 
evaluate the form and content of the text, to make appropriate comparisons 
to other texts, to draw on a repository of contextual knowledge, and to pose 
pertinent questions with regard to the meaning of the text and its implica- 
tions. We assume that the abilities inherent in critical reading are valuable 
both because they allow the reader to engage with the literary text and 
because these skills contribute to an intellectual maturity that can transfer to 
other dimensions, both professional and avocational, of the student’s life. 

Given that importance, however, it becomes all the more urgent to ask in a 
systematic fashion whether specific teaching strategies are succeeding in build- 
ing these capacities. 

The Critical Reading of Foreign Literature in American Higher 
Education 

Teaching college students to read literary texts in a non-native language is 
a core component of traditional understandings of a liberal arts education. 
Foreign language faculty typically regard familiarity with major texts and an 
overview of literary history within specific national traditions as the defining fea- 
tures of a proper undergraduate curriculum for department majors. Readings of 
literary texts certainly define the agenda of graduate instruction, where the 
future teachers of undergraduates are trained, and an ability to read literature 
is the touchstone by which faculty in these fields are recruited, evaluated and 
promoted. Much is therefore at stake for the college learner in the project of 
reading literature, with the supplementary challenge of doing so in a non- 
native language. 

Yet the literary fields rarely attempt to define reading (in contrast to decades 
of debates over definitions of literature).-^ In fact, the seeming familiarity of the 
goal of reading foreign literature is deceptive. A systematic clarification of its 
component parts remains elusive, a situation that is hardly conducive to meet- 
ing the contemporary challenges to higher education. Liberal arts educators 
need to be able to pose distinct questions: What is literature? What is reading? 
What cognitive capacities are necessary to engage with the “foreign” material 
of another culture? And how does “reading foreign literature” become specifi- 
cally “critical reading”? Bringing these distinct inquiries into discussion with 
each other is crucial for the future vitality of college education. 

Several variables overlap, yielding a complex challenge for the instructor 
who should pursue pedagogical strategies designed to enhance student learn- 
ing along multiple axes simultaneously. Clarifying the problems in teaching 


310 



READING FOREIGN LITERATURE CRITICALLY: DEFINITIONS AND ASSESSMENT 


and learning would be important in any case; it is all the more so now, due to 
multiple pressures on this sector of liberal arts education: declining enroll- 
ments in some foreign languages traditionally regarded as commonly taught 
(French and German, but not Spanish); declining levels of L2 (second lan- 
guage) skills for students entering college, producing low enrollments in inter- 
mediate or advanced classes; prcprofessional attitudes among college students 
and their parents, generating resistance toward liberal arts sectors not obvious- 
ly linked to career prospects; economic pressures on institutions of higher edu- 
cation leading to greater scrutiny of low enrollment areas; and a general turn, 
in higher education and elsewhere, toward evidence of accountability and 
assessment. How does the project of teaching college students to read foreign 
literature fare in this environment? 

To answer this question requires us to identify the separate cognitive ele- 
ments that make up the reading of foreign literature in the context of a col- 
lege education. The obvious gateway is, of course, basic familiarity with the 
target language, including grammar, syntax, idiomatic usages, and sufficient 
vocabulary. For the commonly taught languages, much of this should be 
acquired during the first-year course. At least in this sector of higher educa- 
tion, there are commonly accepted criteria for assessment, the American 
Council on the Teaching of Foreign Language (AGTFL) standards. Nonethe- 
less, instructors of higher level courses regularly encounter undergraduates 
who still face difficulties, particularly when — as is frequently the case in liter- 
ary texts, but not only there by any means — language use becomes complicat- 
ed. The point, however, is that teaching foreign literature is never surgically 
separated from teaching foreign language. The organizational model of lan- 
guage learning prior to literature reading, with those two tasks frequently 
assigned to different classes of teaching staff, is flawed, insofar as student lan- 
guage learning ought to continue in the advanced classes and instructors of 
those classes ought to recognize their own responsibility as language teachers 
as well as literature teachers. Instead of separating language and literature 
learning from each other, we need to develop a holistic sense of the student’s 
learning development.” 

A second dimension concerns the character of the complex linguistic texts 
themselves with which students are asked to engage. While designating those 
texts as “literature” appears conventional enough, even this entry point 
requires investigation. With the widespread reconsideration of the literary 
canon, both as part of methodological shifts in literary studies and as a func- 
tion of the politicized “culture wars” of the 1 980s and 1 990s, the selection of 
texts utilized in the college curriculum has been modified. 4 While some of this 
involves shifting from one set of authors to another in order to develop more 
inclusive reading lists, e.g, by increasing the number of texts by women writ- 
ers and minorities, this curricular reshaping has also included a reduction in 
the amount of course time dedicated to explicitly literary material, replaced — 
in the spirit of cultural studies — by other sorts of texts, including documents 
that might otherwise belong to disciplines such as history, philosophy, or poli- 
tics. Indeed, such courses can also include reading journalistic accounts of 
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current events. This is not the place to reopen debates over the relative value 
of narrowly literary versus broadly cultural curricula. The point is that the 
reading of foreign literature, strictly understood, is one specific activity, distin- 
guishable from other foreign language reading that might be included in the 
course of study for the college student in a “foreign literature” course. 

Although we can distinguish literary reading from engagement with other 
linguistic texts (philosophical, historical, etc.), it is important to remember that 
the undergraduate student is probably less of a purist on this matter. On the 
contrary, the undergraduate with intermediate or advanced skills in L2 is likely 
to be as interested in the history, politics, and general culture of that language 
community as he or she may be in the literature. Indeed literature may not 
even be the primary interest of the major, who may have some broader affini- 
ty for the culture. This is worth noting, particularly because the faculty mem- 
ber, in contrast, typically has professional interests much more focused on liter- 
ature, even though he or she is sure to also have considerable knowledge of 
the other cultural dimensions of the region. In other words, the professor of 
German literature, dedicated to specifically literary scholarship, is normally 
broadly familiar with German history and politics, topics probably of consid- 
erable interest to the student; an effective curriculum would build on that stu- 
dent interest by providing opportunities to engage in other, non-litcrary sorts 
of reading. The result can only be an enhancement of language skills, which 
additionally might well serve the faculty member’s goal of engaging students 
more intensely with literary texts. 

It is also worth noting that even within the specific terrain of strictly liter- 
ary reading (despite the importance of other sorts of reading as vehicles to 
enhance the student’s growth), there remains an element of tension between 
student learning needs and the literary or, more precisely, literary-historical 
orientation of the faculty instructor. It is important to identify this discrepancy 
in order to develop appropriate strategies to enhance student reading. Beyond 
introductory language acquisition courses, foreign language curricula in the 
commonly taught languages (and in English as well) are typically organized in 
terms of a literary-historical sequencing; departments structure undergraduate 
courses in order to provide foundational coverage of literary-historical devel- 
opment. Frequently, if not always, this coverage is provided through a series of 
chronologically organized courses. While such linear organization mirrors the 
understanding of the disciplinary study of the literary field, it largely ignores 
student learning needs. In other words, what frequently comes first in the 
course is the historically earliest material, which is not necessarily the sort of 
text which would best support student engagement with literature, a poor 
choice as a first step in a hypothetical scaffolding of increasingly challenging 
texts. Curricular planning with a learning goal of enhancing student reading 
skills would do well to organize the trajectory of courses in terms of student 
learning development rather than solely in terms of historical sequence. We 
need to recognize that the discipline’s understanding of literary-historical 
development is not the same as the development of student learning; the chal- 
lenge we face involves redesigning course structures — and the articulation of 
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courses across the major — in terms of student development and learning 
needs rather than retaining course organizations that reflect the disciplinary 
assumptions of the respective fields. 

In this context, we can only begin to outline some of the relevant curricu- 
lar choices. The traditional paradigm of curricular organization in literary 
fields — which still retain considerable credibility — involves an aspiration to his- 
torical coverage. For all of the innovation in canon expansion, a fundamental 
historicism remains in place: while the content of literary periods may have 
changed dramatically, the compulsion to cover each period — and typically in 
chronological sequence — tends to define the organization of instruction. What 
holds on the departmental level for curricular structures holds on the course 
level as well for each syllabus, structured to reflect the historicist dictates of 
research rather than the learning needs of students. What would the latter 
suggest? How might one restructure literary learning to emancipate the devel- 
opment of student reading skills from the presumed trajectory of (national) lit- 
erary-historical patterns? It would mean giving primacy to the project of 
building student skills in reading (in its multiple dimensions: syntactic, inter- 
pretive, critical, etc.) and selecting as well as sequencing texts with pedagogical 
needs in mind, even at the risk of ignoring literary historical coverage (Barr 
and Tagg). It would mean moving from simpler to more complex texts and 
making selections specifically designed to teach particular reading skills rather 
than to cover a historical period. Not surprisingly, any such loosening of the 
constraints of historicism is likely to shift attention to various formal ele- 
ments — stylistic range, rhetorical capacities, genre variation. The point of 
such a rediscovered formalism, however, would not be an insistence on aes- 
thetic autonomy — as salutary as such an agenda might be — but the impor- 
tance of focusing on reading skills and therefore on textual selections con- 
ducive to building these skills. 

We have so far identified several frames around the critical literary read- 
ing of foreign literature: basic L2 acquisition, the competition between literary 
and non-literary texts, and the relationship of enhancing reading skills to the 
presentation of larger literary-historical narratives. One last dimension 
requires our attention, and that is the reading of foreign literature as a func- 
tion of students’ ability to read literature in their native language. Surely col- 
lege students trying to read foreign language literature will have previously 
read literature in their own language; indeed students are very likely to have 
taken college-level courses in which literature has been taught and read. One 
must therefore factor in student’s native-language literary reading abilities in 
order to develop an appropriate foreign literature pedagogy and to understand 
a further element of the assessment discussion: to what extent is the student’s 
ability to read literature thoughtfully and critically a function of the literature 
instruction the student has previously received, typically from an English 
department curriculum or from general education courses? Ideally, foreign lit- 
erature curricula would build intentionally on other courses that contribute to 
an enhancement of student reading ability and pursue a self-conscious articu- 
lation with other literature courses. 
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Reading foreign literature is linked to a student’s versatility in reading 
native-language literature; reading foreign literature, however, adds another 
dimension — not simply the additional language barrier but the imperative to 
consider the cultural difference between the native culture and the cultural con- 
text of the text. For the student, engaging with the foreign literary text is also 
an encounter with another culture, a dimension which is never fully absent 
from reading a native-language text (which might come from another era or 
region), but this gap is much more pronounced when the student reads a for- 
eign literary text in another language. The reading of foreign literature requires 
a cultural hermeneutics, the capacity of a student to recognize hypothetical 
cultural differences and to integrate them into his or her textual interpretation. 
This particular interpretative obligation constitutes a key element of the critical 
reading of foreign literature, which could be an explicit learning goal of the 
undergraduate major in the appropriate fields. Among the benefits of studying 
foreign literatures and cultures, the development of cross-cultural interpretive 
capacities surely figures as one with considerable practical ramifications. 

Critical reading of foreign literature includes not only that cross-cultural 
perspective, but also the need to respond thoughtfully and interpretively to lit- 
erary texts that are frequently richer and more polyvalent than the straight- 
forward writing of journalism or the social sciences. Reading foreign literature 
is, in other words, both about reading literature and about reading a foreign 
tradition. Building student skills in these particular dimensions can be at odds 
with literary-historical or other theoretical paradigms. The goal of an under- 
graduate curriculum focused on learning needs would emphasize building 
reading skills, in particular reading foreign literature, rather than notions of 
literary-historical coverage or any other paradigm which would impose a theo- 
retical constraint on the engagement with students. Thus reading skills are a 
core goal of liberal arts education that will serve students well; the critical 
reading of foreign-language original material builds this mental capacity in an 
especially salient way. Yet once we have identified the location and particular 
importance of critical reading as a goal for foreign literature departments, 
how can we assess our curriculum? 

Assessing Critical Reading Skills 

Understanding and articulating what students should be able to demon- 
strate as a result of instruction is the first step in assessing curriculum. For this 
to occur it is important to obtain consensus from faculty members in the pro- 
gram as to the specific behaviors that constitute mastery of critical reading. 
Paramount to this first step is making sure that relevant instruction is developed 
and provided so that students may attain these goals. This process presents fac- 
ulty with the opportunity to ask whether conventional (and perhaps often unar- 
ticulated) assumptions regarding learning goals remain appropriate or whether 
they should be revised in light of changing circumstances, such as lower levels 
of foreign language preparation of entering college students, changes in second 
language acquisition strategies, or shifts in the general education environment 
where students may have developed their basic skills in reading literature. 
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For tasks like critical reading, devising or selecting assessments can be 
challenging because of the latent nature of the processes involved in con- 
structing meaning from texts. Because reading behaviors such as comprehen- 
sion are theoretical constructions, they must be inferred through other behav- 
iors, usually secondary to the reading task itself, such as writing, responding to 
multiple choice questions or engaging in academic discourse about the mean- 
ings and interpretations of texts. 

Because of the complexities associated with measuring an unobservable 
behavior such as critical reading, reading researchers in the humanities have 
looked to the works of measurement scholars to inform assessment practices. 

A similar approach should be followed for the foreign languages. Although fre- 
quently tests are selected based on their presumed objectivity, ease of adminis- 
tration, and scoring, good assessment begins with answering questions about 
how test results will be used. This is validity consideration. 

According to Samuel Messick, “Validity is an integrated evaluative judg- 
ment of the degree to which empirical evidence and theoretical rationales sup- 
port the adequacy and appropriateness of inferences of test score interpreta- 
tion” (749). Validity is not a property of the test itself. A test may be valid for 
some intended purposes but not for others. This perspective places responsibil- 
ity on the test consumer to understand the particular purposes of tests and 
their relative strengths and weaknesses. Almost any kind of information about 
what is being assessed can contribute evidence as to the meaningfulness of test 
score interpretation. Validity does not require any particular form of evidence; 
however, it does require justification that the evidence collected validates test 
use. In the realm of validity, considerations for curricular assessment of critical 
reading in a foreign language should include at minimum the evaluation of 
content knowledge and representativeness and construct validity. 

A key issue in constructing or selecting an assessment task is setting up 
boundaries for the content domain of interest. An evaluation of content rele- 
vance should involve an appraisal of how well the assessment task provides a 
representative sample of what is emphasized in the program’s curriculum. 
Understanding and articulating what students should be able to demonstrate 
as a result of instruction is the first step in assessing curriculum. Clearly stated 
learning objectives are a requirement for effective assessment. Operationalizing 
behaviors involves coming up with specifications and describing how students 
are expected to perform on a domain of critical reading tasks that the assess- 
ment is supposed to represent. This requires a thorough understanding of the 
program’s content, identifying what is being taught and what domains are 
being emphasized or given priority in the curriculum. Learning objectives 
should be clear, realistic, connected to what is being emphasized in the cur- 
riculum, and guided by current learning theory and reading research (Linn 
and Gronlund). 

In addition to content considerations, understanding exactly what an 
instrument is measuring is critically important. All assessments of critical read- 
ing will be no more than samples of behaviors that approximate critical read- 
ing in the foreign language. This poses the question of what we believe actual 
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critical reading behavior to be. While reading researchers have struggled with 
how to define critical reading and its components, measurement within this 
field seems to be focused on comprehension. Because of these complexities, 
serious concerns may arise when developing or selecting an instrument that 
may either fail to capture important aspects of the behavior to be measured or 
the chance that test performance possibly could be influenced by other behav- 
iors irrelevant to the task. According to Linn and Gronlund, construct valida- 
tion “is the process of determining the extent to which performance on an 
assessment can be interpreted in terms of one or more constructs” (83). 
Whenever selecting or developing an instrument to assess critical reading, care- 
ful consideration should be given to the relative strength and weakness of dif- 
ferent assessments, including what they capture and what they fail to capture. 

Assessment tasks used to evaluate foreign language critical reading have 
largely taken the form of traditional comprehension questions, requiring stu- 
dents to respond to items by either selecting a correct answer or by construct- 
ing their own responses in cither a written format or verbally. Because of their 
flexibility, multiple choice tests are the most prevalent of the fixed response 
formats. They are easy to administer and to score and they measure compre- 
hension behavior at the reading level only, as opposed to open response for- 
mats that require that the respondent be able to write. They are also more 
reliable due to their objective nature. However, these formats are limited in 
that they weigh heavily upon factual knowledge, often measure product 
instead of process, require a single correct answer, do not allow for the con- 
struction of knowledge (which is central to how cognition and learning theo- 
rists view reading), and can be susceptible to guessing if items are cue depend- 
ent or if item distracters are not at least as plausible to the respondent as the 
correct answer (Linn and Gronlund 200-02). 

In contrast, constructed response tasks, such as essays, oral exams, and 
even tests of recall eliminate guessing and allow for more nuanced answers, 
which is more appropriate for measuring critical reading as students are free 
to interpret, construct, and relate ideas. However, results of such assessments 
are limited by student achievement in other skills, such as reading, speaking, 
and recall, and therefore deflect attention away from the skill under investiga- 
tion — critical reading. Also, due to the subjective nature of these assessments, 
they are more time consuming to score and less reliable as rater inconsisten- 
cy — due to lack of understanding of the scoring criteria — can limit the value 
of the results unless care is taken to develop standardized scoring procedures 
such as rubrics and rater training. 

In between fixed response formats and complex assessments (such as the 
essay test) are short answer items like the cloze task, which requires students to 
provide one word substitutions for missing words in a text. Although these 
tasks require students to supply answers, which thus eliminates guessing, these 
tests have been criticized because they assess reading comprehension at the 
most basic level (i.e., at the sentence level instead of at a global level) 
(Shanahan et al. 229-55). 

The weakness of the assessment procedures mentioned above do not limit 
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the utility of assessing student performance. More valid assessments will result 
with a better understanding of what critical reading actually is and will require 
the help of reading researchers and humanities scholars to define it compo- 
nents. For now, however, we must be mindful of the approaches we select, 
keeping in consideration how we are going to use and interpret results. This 
includes approaching assessment through a reliance upon multiple measures. 
When these cautions are taken, assessment can improve instruction, mold cur- 
riculum, and inform faculty about their teaching. Assessment can enhance stu- 
dent learning by clarifying learning goals and providing feedback to students 
about their progress so that they can be actively engaged in learning. When 
conceptualized in this way, assessment becomes part of the teaching and 
learning process. 

In summary, improving student reading skills has long been an implicit 
goal of a college education. In the context of current challenges to higher 
education, it is urgent to develop effective rationales and descriptions of these 
skills as well as appropriate assessment strategies to measure them. Developments 
in education research can support liberal arts faculty in articulating learning 
goals and pursuing them effectively. This holds for foreign literature reading as 
much as for other parts of the liberal arts. 

Clarifying liberal arts education success in foreign literature critical read- 
ing is crucial for several reasons. Most directly this process can provide feed- 
back to foreign literature instructors and promote reflection on existing peda- 
gogy and elicit change where appropriate. Since numbers of majors in all the 
foreign literature fields remain weak, improved quality of instruction can only 
prove salutary. In addition, however, critical reading of foreign literature 
entails a project that is networked in important ways with desiderata elsewhere 
in the humanities: reading non-foreign literature (typically the domain of 
departments of English), and non-literary reading which is central to other 
humanities fields, notably history. Each reading zone has its own challenges 
and peculiarities, but given the importance of reading, this discussion suggests 
the need for a cross-disciplinary strategy of undergraduate reading instruction. 
Rather than losing ourselves in territorial debates over who owns reading, 
humanists would be well advised to collaborate across traditional disciplinary 
and departmental divisions in order to promote the urgency of a reading 
agenda, even when such a program may challenge conventional but outdated 
pedagogical strategies in humanities departments. 

Yet this project goes beyond the humanities majors, even if research sug- 
gests that humanists are more likely to advocate for building reading skills than 
scholars in other areas (where numeracy may have greater importance). The 
ability to read well — closely, thoughtfully and interpretively, which is to say, 
critically — is a fundamental capacity and a key element in the intellectual 
maturity that the public expects of students with a liberal arts education. 
College curricula should be able provide strong reading skills to all undergrad- 
uates, not only to humanities majors. Whether an enhanced reading pedagogy 
takes place in separate general education curricula or in transformed peda- 
gogy across the curriculum or in both is a question that goes beyond the scope 
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of this paper and is in any case likely to be decided in different ways at differ- 
ent institutions. However a robust reading curriculum develops, we are certain 
that foreign language and literature instruction can play a crucial role in it, 
and for it to succeed, the complexity of assessing critical reading will have to 
be addressed. 


NOTES 

1 “Freshmen have never arrived at college with impressive writing skills. 
Even in the 1890s, when only a tiny, privileged minority went to college, a 
distinguished visiting committee concluded that ‘about 25 percent of the 
students now admitted to Harvard are unable to write their mother- 
tongue with the ease and freedom absolutely necessary to enable them to 
proceed advantageously in any college course.’ Since then, the problem 
has become more serious ...” (Bok 82). 

~ At stake is the tension between the study of literature (literary history, in 
particular) and the enhancement of literacy (reading skills). The seminal 
account of the problematic is Walter J. Ong’s Orality and Literacy: The 
Technobgizing of the Word. Cf., Russell A. Berman, Fiction Sets You Free: 
Literature, Literacy and Western Culture. The efforts to define (or to expand the 
definition of) literature are immense, cf., Jean-Paul Sartre, “ What is 
Literature?” and Other Essays or David Damrosch, What is World Literature? 
The conceptual gap between literature and reading corresponds to the 
distance between a curricular approach based on content and (historical) 
coverage, on the one hand, and student learning needs, on the other, i.e., 
ultimately the fault-line between, in traditional terms, research and teach- 
ing. The systematic undervaluation of teaching and learning reflects a 
literary-historicist bias that threatens to undermine the effectiveness of 
liberal arts humanities education. 

Q 

J Cf. Modern Language Association, “Foreign Languages and Higher 
Education: New Structures for a Changing World”; and responses to the 
MLA report in The Role of Reading in Reconfiguring Foreign Languages Programs 
in the journal Reading in a Foreign Language. 

^ These debates over cultural values and their role in higher education gen- 
erated extensive publications, including Allan Bloom, The Closing of the 
American Mind', James Davison Hunter, Culture Wars: The Struggle to Define 
America ; and Gerald Graff, Beyond the Culture Wars: How Teaching the Conflicts 
Can Revitalize American Education. 
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THE COLLABORATIVE WORLD LANGUAGES 
DEPARTMENT: A TEAMWORK APPROACH TO 
ASSESSING STUDENT LEARNING OUTCOMES 

JOSE G. RICARDO-OSORIO 


A RANKING OF EXCELLENCE (in the purest pedagogical sense) of any 
program at any institution of higher education should be closely connected 
with the learning that the students may attain. However, many specialized 
publications such as US News & World Report base the ranking of colleges and 
universities on their financial resources, their activities, and the scholarship 
production of their faculty. Nowadays, with the advent of the assessment 
movement and pressure from the government, policymakers, accrediting agen- 
cies, and stakeholders, higher education institutions have been forced to make 
a shift in their rhetoric of excellence. The foci of institutional excellence are 
now centered on student learning and the value added of a college education 
(Nusche 5). 

Under this view, academic programs are feeling the impact of external 
scrutiny. Therefore, it will no longer be valid to say that an institution of high- 
er education is academically outstanding if proof of student learning is not 
reported and/ or the attainment of learning is not demonstrated. Many critics 
of the current higher education system question the real value of a college 
diploma if evidence of learning is not indicated through programmatic assess- 
ment plans. Notwithstanding, some faculty and administrators are resisting 
vehemently the idea of documenting student learning through such measures 
(Suskie 37). I witnessed this sentiment at the 2007 Philadelphia Modern 
Language Association (MLA) Delegates Assembly discussion on student learn- 
ing outcomes assessment. Some attendees voiced their concerns as they view 
the assessment movement as an affront to academic freedom or as a hidden 
agenda of the government to legislate higher education. The reality is that the 
foreign language and literature profession is evolving and we need to embrace 
this change collectively. 

In the twenty-first century, language and literature programs are to be 
more cohesive and faculty are to work together in assessing student learning 
outcomes (MLA 3). In response to this call for reform, the foreign language 
and literature professoriate must shift from the current individualistic/ depart- 
mentalized modus operandi to a more collegial and collaborative work environ- 
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ment. As a unit, language and literature faculty must look into their pedagogi- 
cal practices by positing the following questions: 

• How effective are language/literature programs? 

• How current are our programs? 

• How do we know that our students learn what we teach them? 

• Do we actually know if they are learning a second language 
and/ or literature? 

• How well are they learning? 

• Are our graduates in foreign languages/literature fully prepared 
to join the workforce of the twenty- first century? 

To obtain answers to these questions, a collective department assessment plan 
needs to be developed. 

The assessment plan must be motivated by the natural inquisitive mind 
that we all possess (the same one that drives our scholarly interests). World lan- 
guages departments should not let the five-year report or the accreditation 
reviews (and in some cases, professional organizations) dictate the implementa- 
tion of an assessment plan. We are all familiar with the dean’s letter in which 
we are told to create an assessment plan and/ or submit results. Such demands 
should not be the raison d’etre for student learning outcomes assessment. There 
is no reason to claim that our academic freedom is at stake if we only think of 
assessment when the administrators “turn on the assessment switch.” 

If world languages departments want to regain the relevance that they 
have lost (Holquist 6), a good start could be to take a closer look at their pro- 
grams by examining student learning outcomes. At the end of the day, we will 
not get the recognition we deserve if we insist, stubbornly, on not demonstrat- 
ing that our students do learn a second language and that our rigorous litera- 
ture/ culture courses do indeed prepare them to think critically. We all know 
the ancient African proverb, “It takes a village to raise a child.” In the context 
of this discussion, the village is our language department and the children are 
our students. It goes without saying that assessment of student learning is a 
complex endeavor. But it is doable. All department constituencies must act as 
chief academic officers (CAOs) of their stakeholders (the students), and assess- 
ment of learning should be regarded as a means to provide progress reports to 
the stakeholders. The process of assessing students’ progress must be faculty 
driven. To gain ownership of the assessment initiative, we should take into 
consideration the American Association of University Professors’ (AAUP) 
guidelines regarding assessment: 

The faculty should have primary responsibility for establishing 
the criteria for assessment and the methods for implementing it. 

The assessment should focus on particular, institutionally deter- 
mined goals and objectives, and the resulting data should be 
regarded as relevant primarily to that purpose. To ensure respect 
for diverse institutional missions, it is important that uniform 
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assessment procedures not be mandated across a statewide system 
for the purpose of comparing institutions within the system. The 
assessment process should employ methods adequate to the com- 
plexity and variety of student learning experiences, rather than 
rely on any single method of assessment. To prevent assessment 
itself from making instruction and curriculum rigid, and to 
ensure that assessment is responsive to changing needs, the instru- 
ments and procedures for conducting assessment should be regu- 
larly reviewed and appropriately revised by the faculty. (1 1) 

In this essay, I will present information that could guide a coherent learning 
outcomes assessment plan for world languages and literature. Some of the rec- 
ommendations outlined herein are already being implemented at many world 
languages departments. If that is the case, I invite you to share your experi- 
ences via the appropriate professional groups within the MLA (or any other 
professional organization).-^ 

Culture of Assessment and Assessment Literacy 

A culture of assessment and assessment literacy are integral parts of a 
successful student learning outcomes assessment plan. The National Center 
for Postsecondary Improvement (NCPI) considers that an institution truly has 
a culture of assessment if: 

1 . There is campus-wide support for student assessment. 

2. The goals for student assessment are well-delineated. 

3. There is a strong allocation of resources for student assessment. 

4. The student assessment plan is part of the institutional assessment plan. 

5. Academic affairs and student affairs constituencies are acting 
members of the student assessment plan. 

We can say that there is a culture of assessment when all the participating 
constituencies embrace assessment (faculty buy-in is especially important) and 
foster its application for a programmatic improvement of student learning 
(Suskie 37). In a world languages department, the culture of assessment is 
defined by the attitudes and dispositions of faculty members towards evalua- 
tion of student progress. Thus, assessment is not regarded as an external 
imposition, but as an internal, self-initiated method to gauge the quality of 
learning. Language and literature faculty use assessment to delineate and imple- 
ment curricular changes to improve student learning of the target language. 

The culture of assessment goes in tandem with assessment literacy, that is, 
the knowledge base about assessment and data analysis that faculty may have. 
Assessment literacy is a “tool for analyzing and reflecting upon test data in 
order to make informed decisions about instructional practice and program 
design” (Boyles 1 8). For example, everyone assessing a section of a specific 
course should be able to interpret assessment results and draw conclusions 
about their curricular implications. As leaders, department chairs must spear- 
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head plans to promote the culture of assessment and assessment literacy. It 
could be a road block to assessment if the department chair is one of its 
biggest opponents or when the chair takes a back seat in the assessment plan- 
ning because his or her expertise is in literature or linguistics and not in edu- 
cation. Renee Waldinger suggests that a department chair “may initiate a 
plan, provide the resources for its development or promote it within the 
department as well as without ...” (49). I have had conversations with many 
department chairs who do not seem to know what is going on in their depart- 
ments regarding student assessment. While faculty with education degrees 
have a functional knowledge of assessment, many college faculty do not, and 
therefore, they have to learn this new skill (Suskie 39). However, a degree in 
education is not necessary to understand and implement student learning out- 
comes assessment. All that is needed is an open mind to see our educational 
mission under a different lens. 

The culture of higher education tends to center around autonomy and indi- 
vidualism. To survive the winds of change currently facing academia, a more 
cooperative stance is necessary (Uchiyama and Radin 272). Thus, one of the 
objectives of creating a culture of assessment and becoming assessment literate 
is to focus faculty’s attention on a common goal: student learning. This may help 
narrow the gap that exists between language and literature faculty, especially at 
research institutions with large programs. In a recent research study, Richard 
Donato and Frank Brooks reported that there are pedagogical discrepancies 
between language and literature courses (184). Language and literature faculty 
have different agendas. The former group wants their students to become profi- 
cient in the target language whereas the latter wants students to develop their 
capacity to read and think about literature. Literature courses do not incorpo- 
rate language learning outcomes within their curriculum (197). If learning out- 
comes are set up individualistically, students may be exposed to a fragmented 
curriculum that may not lead to advanced language learning (196). We are all 
familiar with complaints from faculty regarding the poor linguistic skills of stu- 
dents in literature courses. Often times, this weak performance may be due to a 
poorly articulated language program. In literature courses, students are required 
to use both higher-order thinking and advanced language skills. Therefore, 
learning outcomes for novice and intermediate levels should introduce foundation- 
al linguistic skills to prepare students for advanced language and literature courses. 

In the language/literaturc gap, there is also a hierarchy pertaining to 
teaching assignments that may affect the outcomes and/ or assessment of stu- 
dent learning. At some institutions, adjuncts or teaching assistants (TAs) are 
charged with teaching language classes whereas tenured professors teach 
uppcr-level/literature courses. Again, if there is no communication between 
these two groups, learning outcomes assessments may be poorly aligned or 
even worse, badly defined. For instance, if lower-level courses emphasize the 
knowledge of grammar (usually assessed with multiple-choice tests) and middlc- 
and upper-level courses take a dramatic shift to oral production and discourse 
analysis, students entering such courses will not fare well. Under such circum- 
stances, any language proficiency will come to naught. 
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In a collaborative department, both language and literature faculty must 
work as a team when putting together and implementing a student learning 
assessment plan. The success of this work will lie in the ability to create a 
cohesive plan. It is important that adjuncts and TAs be included in the assess- 
ment initiative. Department chairs must establish working teams in which 
more experienced faculty mentor junior faculty. In addition, the culture of 
assessment and assessment literacy initiatives should be part of a faculty-driven 
professional development plan. Faculty are more receptive to undertake an 
assessment plan and gain ownership of it if they are given the opportunity to 
draft the plan and carry out its implementation following a timetable estab- 
lished by them (Khattri et al. 11 7). Therefore, an effort should be made for 
faculty to initiate, plan, implement, and evaluate any departmental assessment 
endeavors. The chair or the assessment coordinator should serve as a facilita- 
tor of the process. 

It is important to identify the purpose of the assessment plan before 
undertaking a specific project. In general terms, an assessment plan may have 
different purposes: classroom assessment, course assessment, program assess- 
ment (see fig. 1). Whatever the purpose is, faculty ownership is advised. If the 
goal of the assessment plan is to gauge student learning after an instructional 
unit has been covered (classroom assessment), the assessment plan must be tied 
to goals (or standards) for language proficiency and literature stated by the 
department. Very few foreign languages departments at the undergraduate 
level are using the American Council on the Teaching of Foreign Languages 
(ACTFL) Standards for Foreign Language Learning and the Proficiency Guidelines as 
frameworks to devise assessment for language proficiency (Ricardo-Osorio 
599). In a successful plan, departmental teams need to decide whether to use 
the standards and guidelines proposed by ACTFL or to create their own. It 
goes without saying that the latter option is labor intensive. Thus, I advise 
departments to incorporate the ACTFL criteria. 2 It is also ideal to use some of 
the recommendations stated in the 2007 MLA Report, “Foreign Languages 
and Higher Education: New Structures for a Changed World.” 

When it comes to assessing only literature, more work might be required 
since learning outcomes for literature may be more difficult to devise. Efforts 
should be made to delineate specific standards for literature. To date, there are 
no established educational guidelines for the teaching and learning of litera- 
ture, let alone its assessment. When I ask at professional conferences what the 
role of literature is within the curriculum or which learning outcomes the stu- 
dents must demonstrate, I get different answers. Even among students we have 
heard complaints about having to take many literature courses. Students do 
not understand the importance of literature in the liberal education curricu- 
lum (or we do not give them a clear notion). To address this lacuna, literature 
and language teachers must define specific learning outcomes and standards of 
proficiency for literature courses. The study of literature should not be devot- 
ed solely to upper-level courses. Literature should be introduced at intermedi- 
ate levels as part of the cultures or connection standards established by ACTFL. 
Departments should set out clear criteria explaining the role of literature in 
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Fig. 1. Learning Outcomes Assessment Chart, Department of Modern 
Languages, Shippensburg University of Pennsylvania, date unknown. 


the students’ foreign language learning experience (MLA 2). 

To see the connections among language and literature courses/ sections 
within a program, a curricular matrix should be designed (see fig. 2). A cur- 
riculum matrix is an advance organizer that includes information on course 
offerings (in a sequential way), key learning outcomes and key assessments for 
each course (Allen 42). Information is presented in a table format. This way, 
faculty can see a layout of the curriculum and make the necessary adjustments 
in terms of assessments and/ or student learning outcomes. Departments with 
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Course 

Learning Outcomes 

Key Assessments 

Language 

Implications 

Operational 

Information 

SPN 202: 

Intermediate 

Conversation 

a. Identify key information 
from authentic materi- 
als to be used in oral 
tasks. 

b. Provide and obtain 
information orally 
using appropriate com- 
municative strategies. 

'Success is evident, if 
outcomes are performed 
as described in the 
ACTFL Proficiency 
Guidelines for the inter- 
mediate-high level. 

a. Integrated 
Performance 
Assessment (IPA). 

b. Role play of situations 
depicting simple 
transactions. 

Students will interpret 
authentic material to 
obtain information. 

Then, they will exchange 
information with a part- 
ner. Finally, they will 
give a presentation 
based on the informa- 
tion obtained from the 
reading and the interac- 
tion with their partners. 

Faculty teaching SPN 
103, 202, and 204 
should work together 
designing the key 
assessments and 
rubrics. Once the key 
assessment is ready, fac- 
ulty teaching 202 will 
select a random sample 
from the student popula- 
tion taking 202. These 
students will take the 
assessment. This 
assessment will be given 
in the spring 1 (The start 
of the assessment cycle). 

SPN 204: Ideas 
and Culture of 
the Spanish 
Speaking World 

a. Identify key products 
and practices from the 
Spanish speaking 
world. 

b. Compare and contrast 
two products or prac- 
tices from the Spanish 
speaking world. 

a. Essay. 

Students will write an 
essay in which they will 
analyze two products 
and/or practices from 
the Hispanic culture. In 
this essay, student will 
have to support their 
ideas with theory dis- 
cussed in the course. 

Faculty teaching SPN 
202, 204 and 260 
should work together 
designing the key 
assessment and rubric. 
Once the key assessment 
is ready. Faculty teaching 
204 will select a random 
sample from the student 
population taking 204. 
These students will take 
the assessment. This 
assessment will be given 
in the fall 1 (2nd phase of 
the assessment cycle). 

SPN 260: Intro 
to Literary 
Studies 

a. Identify characteristics 
of literary texts. 

a. Integrated 
Performance 
Assessment (IPA): 
Interpretive, 
Interpersonal and 
Presentational tasks. 

Students will interpret 
three types of literary 
passages. Next, they will 
engage in conversations 
about their textual inter- 
pretations. Then they 
will write a summary 
highlighting their main 
characteristics. 

Faculty teaching SPN 
202, 206 and 313 
should work together 
designing the key 
assessment and rubric. 
This assessment will be 
given in the spring II 
(3rd phase of the 
assessment cycle). 

SPN 302: 
Advanced 
Spanish 
Conversation 

a. Narrate events using 
main time frames. 

a. In-house oral inter- 
view (OPI). 

Students will sit for a 
thirty-minute oral inter- 
view. The interview will 
be recorded and then 
analyzed. The non-offi- 
cial interview must fol- 
low the protocols of an 
official OPI. Students are 
expected to be at the 
Advanced-Low level. 

Faculty who have taken 
the ACTFL OPI workshop 
should volunteer to do 
the interview. This way 
all of the students taking 
SPN 302 can be tested. 
This assessment will be 
given in the fall II (4th 
phase of the assessment 
cycle). 

SPN 309: 

Spanish 

Phonetics 

a. Recognize specific 
sounds in spoken 
texts. 

a. Interpretive listening 
activity. 

N/A for this assessment 
cycle. 

N/A for this assessment 
cycle. 

SPN 312: 

Spanish 

Grammar 

a. Use correct grammati- 
cal forms in written 
texts. 

a. Presentational task: 
Replying to a blog 
entry. 

N/A for this assessment 
cycle. 

N/A for this assessment 
cycle. 
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Course 

Learning Outcomes 

Key Assessments 

Language 

Implications 

Operational 

Information 

SPN 313: 
Advanced 
Composition 
and Stylistics 

a. Express ideas logically 
and with some degree 
of sophistication in 
writing. 

a. Presentational task: 
Opinion Essay. 

N/A for this assessment 
cycle. 

This assessment will be 
given in Spring 2011. 

SPN 490: 
Selected Topics 

a. Synthesize information 
from different sources. 

b. Establish connections 
among different fields 
to analyze literary texts. 

c. Demonstrate accurate 
control of the Spanish 
language both orally 
and in writing. 

a. Final research paper. 

b. Oral defense of 
research paper. 

Students will demon- 
strate that they have 
aeguired the knowledge 
and the skills to be 
independent analysts of 
literary texts. 

Data collected from stu- 
dents in the foreign lan- 
guage education pro- 
gram will be used to sat- 
isfy National Council for 
Accreditation of Teacher 
Education (NCATE) 
accreditation. 

This assessment will 
be given in Fall 2011. 


Fig. 2. Spanish (SPN) Major Curricular Matrix, Department of Modern Languages, Shippensburg University of 
Pennsylvania, 2009. 


different language and literature programs should use a curricular matrix per 
program. Once the curricular matrix is devised, faculty should analyze it criti- 
cally by answering the following questions: 


• Does our current curriculum prepare students, from an early 
stage, for advanced language courses? 

• Are the proposed learning outcomes realistic considering our 
course offerings? 

• Are there learning gaps between language courses and literature 
courses in terms of language proficiency? 

• Are our language and literature courses equally balanced in 
terms of offerings? 

• Are the key assessments proposed for language courses congruent 
with the department goals? 

• Is critical thinking properly embedded across the curriculum? 


Collecting answers to these questions (and other questions that faculty deem 
important) and making adjustments accordingly are the first steps for program 
assessment regarding student learning. 

For instance, in my department we created a curricular matrix similar to 
the one presented here. We worked in teams (by language and course level) to 
come up with key learning outcomes, key assessments, and the language impli- 
cations of the assessment plan. The learning outcomes were based on two 
sources: the ACTFL Proficiency Guidelines (to assess language courses) and the 
notes from a departmental discussion on the ideal cognitive domains that the 
students should demonstrate in literature courses. 

Our assessment has been implemented in cycles because it is impossible to 
assess everything at the same time. At present, we are assessing only the 
upper-level courses (i.e., Spanish 313, “Advanced Composition and Stylistics,” 
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and Spanish 360, “Master Pieces of Spanish Literature”). The team that 
worked on Spanish 360 also designed a rubric (see fig. 3) that will be used to 
assess the final research paper. As the assessment has not taken place at the 
time of this writing (it will be done in the fall of 2010), I cannot comment on 
the changes that will occur after the assessment data are collected and ana- 
lyzed, but I would anticipate that some adjustments to the course (or to the 
curriculum) might take place, if the assessment results are not satisfactory. 

I suggest a bottom-up approach when designing the assessment plan. 

This means that the department starts with setting up student learning out- 
comes that serve as program completion benchmarks for the minor, the 
major and general education. These benchmarks will establish the proficiency 
levels and the knowledge base that any undergraduate student should demon- 
strate before leaving the program. At this stage, exit key assessments (also 
called pre-graduation evaluations) should be determined. For example, if the 
department has indicated that students majoring in a foreign language should 
attain an advanced level of proficiency, a comprehensive oral assessment is in 
order (i.e., an AGTFL Oral Proficiency Interview or OPI). If one of the 
learning outcomes for literature is that students demonstrate critical thinking 
by comparing and contrasting two literary texts, detailed guidelines for a 
research paper with an accompanying scoring rubric are to be devised. It is 
worth noting that an assessment plan to gauge program quality might be 
more cost effective if a random sample of students is drawn (Suskie 1 16). It 
will be a daunting endeavor to assess all of the students graduating from a for- 
eign language program, especially at large institutions where foreign lan- 
guages departments offer various languages. 

Note in fig. 2 that my department chose a final research paper and an oral 
defense of the paper as the key assessment for students in Spanish 360. The 
rationale for the research paper as opposed to an essay (i.e., expository or argu- 
mentative) is that the former allows the students to apply more cognitive 
domains (i.e., bibliographical research). The same goes for the oral defense as 
opposed to the usual oral presentation. In an oral defense, the students have to 
back up their claims more thoroughly. A rubric (fig. 3) was created to evaluate 
the paper. Three criteria within the rubric were adopted from the AGTFL/ 
NGATE Program Standards for the Preparation of Foreign Language Teachers since some 
students taking this course are in the teacher education program as well. This 
way, the assessment will serve to collect data for the National Council for 
Accreditation of Teacher Education (NCATE) accreditation purposes. At pres- 
ent, the team is working on the rubric to evaluate the oral defense. The use of 
both assessments will provide relevant data on the students’ attainment of con- 
tent knowledge and linguistic skills relevant to succeed in the 400-level courses. 

When designing the program assessment plan, specific language should be 
included regarding the stages in which the plan will be carried out and the 
cycle in which the learning outcomes will be assessed. As alluded to earlier, it 
is preposterous to assess all of the learning outcomes in a single semester. My 
department’s plan states, for example, that oral proficiency will be assessed in 
phase I (fall semester) and that writing skills will be assesed in phase II (spring 
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Criteria 

Exceeds Expectations (3) 

Meets Expectations (2) 

Not There Yet (1) 

Structure of the Paper 

Thesis clearly given in the intro- 
duction, well developed and lead- 
ing to a convincing conclusion. 

Introduction, development and 
conclusion require some clarifi- 
cation. 

Problems of organization. 
Expression of a sequence of 
ideas without a clear structure. 

Research Content 

Topic relevant and developed with 
details, descriptions and exam- 
ples. 

Topic developed with relevant 
information. 

Topic poorly developed. 

Use of Target 
Language in Written 
Presentational 
Communication 
Standard 1.1 

Use of target language is on tar- 
get when expressing ideas and 
concepts. Accurate and appropri- 
ate vocabulary with very few 
spelling errors. Very few gram- 
matical errors. Meaning is always 
clear. 

Use of target language has 
some inaccuracies, but commu- 
nication is achieved. Appropriate 
vocabulary although somewhat 
limited with fairly frequent 
errors in usage and spelling. 
Some grammatical errors, 
although they do not interfere 
with meaning. 

Use of target language is defi- 
cient. Extremely limited vocabu- 
lary with errors in usage of 
spelling. Writing not compre- 
hensive to native speakers. 
Frequent grammatical errors 
that interfere with meaning. 

Cultural Connections 
Standard 2.1 

The paper presents a relevant 
example of the close relationship 
between the literary text (product) 
researched and a Spanish cultur- 
al perspective discussed in class. 
An explanation of the relationship 
between the literary text and a 
Spanish cultural practice was 
also included. The student pro- 
vided several examples that illus- 
trated these connections. 

The paper presents a relevant 
example of the close relation- 
ship between the literary text 
(product) researched and a 
Spanish cultural perspective 
discussed in class. However, an 
explanation of the relationship 
between the literary text and a 
Spanish cultural practice was 
not included. If included, it was 
not strong. The student provid- 
ed just one example that illus- 
trated these connections. 

The paper does not present a 
relevant example of the close 
relationship between the literary 
text (product) researched and a 
Spanish cultural perspective 
discussed in class. If presented, 
the example is not relevant. An 
explanation of the relationship 
between the literary text and a 
Spanish cultural practice was 
not included. The student did 
not provide an example that 
illustrated these connections. 

Cross-Disciplinary 
Content 
Standard 2.3 

The student drew upon other 
contexts to enrich his/her expli- 
cation of the interrelationship 
among product, practices and 
perspectives. The student used 
researched information of 
Spanish history, geography, visu- 
al arts, music, and film to contex- 
tualize his/her ideas and/or 
examples. 

The student drew upon other 
contexts to enrich his/her expli- 
cation of the interrelationship 
among product, practices and 
perspectives. The student used 
very little researched informa- 
tion of Spanish history, geogra- 
phy, visual arts, music, and film 
to contextualize his/her ideas 
and/or examples. 

The student failed to include 
other contexts to enrich his/her 
explication of the interrelation- 
ship among product, practices 
and perspectives. The student 
used some information of 
Spanish history, geography, 
visual arts, music, and film to 
contextualize his/her ideas 
and/or examples. 


Fig. 3. SPN 490: Selected Topics (Final Research Paper), Department of Modern Languages, Shippensburg University 
of Pennsylvania, 2010. 


semester). The number of phases will depend on the number of outcomes to 
be assessed. 

An easier task is to assess student learning at the classroom level. To this 
end, faculty members must try and use the learning outcomes set up in the 
curricular matrix to devise specific course objectives. In addition, key assess- 
ments for the course must also mirror the assessments proposed in the matrix. 
That being said, faculty must have the freedom to design their own key assess- 
ments. Based on my experience, I recommend for faculty teaching different 
sections of the same course to work together in the design of a key assessment 
to be given at the end of the course. This key assessment could be an Integrated 
Performance Assessment (LPA). An IPA is an assessment procedure that 
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requires students to demonstrate language proficiency in interpretive commu- 
nication, interpersonal communication and presentational communication 
around a single topic or theme (AGTFL, Integrated Performance Assessment 8). 
Thus, students will be exposed to the mechanics of the assessments and will be 
better prepared in the event of being chosen for a program assessment. Faculty 
must provide the learning outcomes and the rubrics for the key assessments to 
the students. The best way to do this is to add the information in the syllabus 
and discuss the assessment plan during class. If samples of previous assess- 
ments are available, these should be shared with the class. This way the stu- 
dents will have a clear picture of what is expected from them in terms of per- 
formance (or demonstration of learning). The same IPA principle can be 
implemented to design activities for the assessment of literature courses. 

Once a key assessment has been administered and data have been collect- 
ed, the assessment team must interpret and analyze these data. The results of 
the assessment (good or bad) should be used to improve the students’ learning 
experience (Suskie 263). For instance, if data from the OPI show that only 
20% of the students majoring in Spanish are obtaining an Advanced-Low 
proficiency rate and the department’s assessment plan stated that at least 80% 
of students should attain Advanced-Low minimum, some major curricular 
overhaul has to be done. This was the case of the results yielded from my 
department’s assessment. To sort out the problem, we encouraged the students 
to spend more time studying abroad and some class instruction was devoted to 
the acquisition of certain linguistic skills required for the advanced proficiency 
level. As the assessment results also suggested changes in teachers’ pedagogy, 
the department’s curricular team worked together and devised a comprehen- 
sive professional development plan. We organized two OPI workshops facili- 
tated by ACTFL. As a result, two faculty members are undergoing training to 
become official OPI testers. 

Just as positive results are celebrated collectively, so too must negative ones 
be tackled as a team (this is not the time for finger pointing). Faculty will be 
more receptive to making and implementing changes if their professional 
principles are not attacked or questioned. 

In conclusion, assessment of student learning outcomes is a complex yet 
doable endeavor that requires an active participation and commitment from 
every single party within the department. A corollary of student learning out- 
comes assessment is the professional forum for discussion that ensues. 
Language and literature teachers (regardless of rank) will be talking about stu- 
dent learning and the curriculum. This time around, the assessment process 
will not be individual, but collective. Learning outcomes will not be delimited 
by a course. On the contrary, they will be framed along a learning continuum. 
The end goal of a team approach to assessing outcomes will be student learning. 


331 



LITERARY STUDY, MEASUREMENT, AND THE SUBLIME: DISCIPLINARY ASSESSMENT 


NOTES 

1 To further your knowledge base on departmental assessment initiatives, I 
recommend reading Linda Suskie’s Assessing Student Learning. 

^ For more information on the standards and the proficiency guidelines, visit 
the publications page on the ACTFL website <http:/ /www.actfl.org>. 
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HOW TO CONSTRUCT A SIMPLE, SENSIBLE, USEFUL 
DEPARTMENTAL ASSESSMENT PROCESS 

BARBARA E. WALVOORD 


SO YOUR DEPARTMENT HAS BEEN TOLD it has to “do assessment.” 
Faculty response may be welcoming or at least open-minded, but perhaps also 
resentful, resigned, or hostile. How should you respond? Or perhaps you have 
an assessment plan, but 

• You have been told it is not good enough; or 

• It is good enough for the assessment people, but seems stupid and 
useless to most department members; or 

• It is useful, but the workload it generates is killing you. 

I have taught literature and writing in English departments both large and 
small, public and private, for over thirty years. In the mid- 1 990s I began to 
speak and write about “assessment,” because I was alarmed, as the national 
movement geared up, and I wanted to see this powerful new force turned for 
good, not harm, to higher education. By now I have consulted and led work- 
shops at more than 400 institutions of higher education, on assessment, teach- 
ing and learning, or writing across the curriculum. I have written a book 
called Assessment Clear and. Simple. 

So in this essay, I want to integrate what I know about teaching literature 
with what I know about meeting requirements for assessment. I want to show 
departments how to construct an assessment system that is 

• Consonant with the requirements of accreditation bodies, boards, 
legislatures, public education systems, or others who are requiring 
assessment; 

• Consonant with the culture and values of the department; 

• Useful to faculty and students; 

• Manageable in terms of time, resources, expertise, effort, and work 
load; 

• Respectful of appropriate faculty autonomy and academic freedom; 

• Not used to punish faculty or make reappointment, promotion, or 
tenure decisions. 
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Beyond that, I want to address the question, “How might we assess our 
most ineffable goals — qualities of mind and heart that we most want the study 
of literature to nurture in our students?” 

But first, the basics of assessment. 

What Is Assessment? 

Assessment is a national reform movement arising from public dissatisfac- 
tion with the perceived inability of undergraduate degree holders to read, 
write, and solve problems. The assumption is that higher education in America 
is broken, and that the way to fix it is to hold faculty and institutions responsi- 
ble for assessing student learning. The assessment movement is exceptionally 
powerful because it has captured the accreditation process, which means the 
accreditors can make us do it. There are many questionable assumptions and 
potential dangers within the movement. When faculty are wary of assessment 
as a movement, I think they are doing their job. 

But there is also good news. The accreditors define assessment as the sys- 
tematic collection of information about student learning for the purpose of 
improving that learning. There is nothing necessarily demonic about that con- 
cept. When rightly done, assessment is actually a good idea. It’s so good we 
can’t not do it. Why would we spend so much time and energy trying to help 
students learn, and then not ask, “Is it working? Are our efforts paying off?” 
We have been doing assessment all along — not as well as we would wish, per- 
haps, and certainly not visibly enough for our constituents, but the essential 
task is to do assessment in a sensible way and then explain it clearly to those 
who need to know. And that sensible way, well explained, will suffice for the 
accreditors. 

Most important, the accreditors leave to faculty the right to define their 
learning goals. So you can find your own ways to articulate the critical think- 
ing, the aesthetic responsiveness, the reflection about meaning and values, the 
awe and self-loss, and the transforming experiences that you hope your stu- 
dents will take away. More about that later, when we get to stating learning 
goals. 

Assessment and Grades 

Meanwhile, a more prosaic question about assessment: the definition of 
assessment might lead you to say, “We give grades — isn’t that assessment?” Yes 
it is. However, the purpose of grades is to answer the question, “How well did 
Mary or Juan do on this assignment/test/course?” The audience for grades is 
future employers or graduate school admissions committees who need to know 
how well that student did. The accreditors are asking the department or pro- 
gram to evaluate the learning of its students as a whole, to answer the ques- 
tion, “What can we as a department do to enhance our students’ learning?” 
The department cannot answer that question simply by saying, “Majors grad- 
uated with an average GPA of 3.6 for courses in the major,” or “Senior grades 
on an essay of literary analysis averaged a B+.” Grades are too blunt an 
instrument for departmental assessment. They do not tell us what to work on. 


336 



HOW TO CONSTRUCT A SIMPLE, SENSIBLE, USEFUL DEPARTMENTAL ASSESSMENT PROCESS 


Instead, we need a more fine-grained analysis that identifies strengths and 
weaknesses, the patterns of growth, or the emerging qualities we wish to nur- 
ture. Faculty conduct that type of analysis as we grade papers and exams, so 
we can use the grading process for assessment; we just have to push back down 
to the finer analysis the teacher has conducted, and we must draw conclusions, 
not about whether this student did well, but about how well the students did 
as a whole. 

You might say, “Well, if students complete English 301, we know they 
have reached the goals, because that is what is required to pass the course.” 
But again, course pass rates do not tell you what to work on. Instead, you 
need, as a department, to know the qualities of your students as a group. 

Many good departments, in fact, conduct this kind of group analysis, at 
least informally. If department coffee pots could talk, they would report over- 
hearing many statements that begin, “I wish our majors were better at ” 

Department meetings or curriculum revision committees often implement 
changes based on faculty perception of students’ strengths and weaknesses. We 
need to ensure that these processes are strong, and we need to explain them to 
the accreditors in language that fits the accreditors’ needs. I have seen many 
departments that were doing a better job of assessment than their written 
reports revealed. 

The Three Steps of Assessment 

Accreditors, no matter how many pages of guidelines they publish, 
require three steps of assessment: 

1 . Articulate goals for student learning in this format: at the end of 
this program, students will be able to.... 

2. Gather information about how well students are achieving the goals. 

3. Use that information to inform decisions and actions. 

Assessment for accreditors requires that you follow the three steps. It does 
not require that: 

• You dumb down the curriculum, leave out your most important 
goals, or state only what is “measureable” in a narrow sense. 

• You rely on “objective” or standardized tests. 

• Everybody teach the same. 

• The accreditor dictates your learning goals for students. 

• You violate academic freedom. 1 

• You use assessment information in personnel decisions such as 
reappointment or tenure. 

Step One: Articulate Goals for Learning 

The first step is for the department to ask, “What do we want our stu- 
dents to be able to do when they finish our undergraduate major/ associate’s 
degree/ doctoral program?” You do not need to worry about whether your 
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statements will be called “objectives” or “learning outcomes” or something 
else. Skip the jargon if you can. Just generate a list of things you want your 
students to be able to do. You will need a list of learning goals for each of 
your courses of study — the associate’s degree, undergraduate major literature 
track, writing track, and each graduate degree. 

The statement of the goals should begin with “The student will be able 

to ” This is very important for your audience of accreditors. The purpose of 

the assessment movement is to get institutions of higher education to move 
from reciting what they do, toward inquiring whether students learn anything. So 
do not include statements such as “The department will offer ...” or “The stu- 
dent will be exposed to ...” or “The student will complete an internship.” In 
the following hypothetical example, drawn from a number of such statements 
I have seen, the department identifies program goals, not learning goals, even 
though the statements imply student learning. 

Example: Inadequate Description of Learning Goals 

The primary goals of the English program are to: 

1 . Teach effective writing; 

2. Help students develop critical thinking and research skills; 

3. Promote a broadened world view through the study of literature; 

4. Foster collaborative learning. 

The accreditor will think that the department doesn’t know the difference 
between focusing on what the department will do and focusing on whether the 
student learns anything from what the department does. So here are the same 
items, turned into “students will” statements. 

Example: Revised Learning Goals, Still Very Broad 

Students will: 

1. Write effectively for a variety of audiences and purposes; 

2. Demonstrate critical thinking and research skills; 

3. Broaden their world view; 

4. Collaborate effectively with other learners. 

The goals above are now stated as student learning, but the goals are still 
broad. They could belong to the sociology or management department as well 
as English. 

Here are some goals (again hypothetical, but drawn from my actual work 
with departments) that specifically mention literature, but they, too, are broad 
because of their verbs. 

Example: Learning Goals with Vague Verbs 

Students will: 


338 



HOW TO CONSTRUCT A SIMPLE, SENSIBLE, USEFUL DEPARTMENTAL ASSESSMENT PROCESS 


1 . Understand the role of literature in expressing and reflecting all 
aspects of human experience; 

2. Understand the concerns and perspectives unique to literary tra- 
ditions and artists; 

3. Discover how literature can assist in understanding ourselves and 
the world around us; 

4. Discover the joy and fun of reading, writing and discussing litera- 
ture. 

The accreditor would probably not jump on the department for these 
goal statements, but at some point, when the department begins to plan how 
to assess these goals, it would need to make the goals more specific. So ask, 
“What are the critical thinking and research skills needed in English, or in a 
particular track of English such as literature or writing?” Then articulate 
those skills. Or ask, “What will students do that will suggest they are broaden- 
ing their world view or understanding themselves and the world around 
them?” 

Here is a hypothetical list of learning goals for undergraduate literature 
majors, again developed from a number of such goal statements that various 
departments have shared with me. 

Example: Learning Goals with More Specific Language 

By the end of the undergraduate literature-track major, students will be 

able to: 

1. Describe and analyze major literary works, literary themes, and 
trends from English, American, and at least one non- Western lit- 
erary tradition. 

2. Identify and analyze the cultural, sociological, ideological, histori- 
cal, linguistic, and other aspects of works of literature. Discuss 
the ways in which literature is a product of its time and culture, 
but also how literature can transcend or critique its culture or 
break new ground. 

3. Analyze and critique literary works, orally, in writing, and in dis- 
cussion with others, using at least two theoretical/ critical 
approaches, and employing tools of literary-critical analysis. 

4. Discuss the complex role of writer and reader/ viewer in the 
mutual creation and enactment of literary work. 

5. Make aesthetic judgments about literature and support them. 

6. Find, employ, and cite sources effectively. 

7. Follow ethical principles of the discipline for collaborating with 
others and for using sources. 

Most departments stick with a list such as the one above that include 
aspects of what I would call “critical thinking,” but not some of the more inef- 
fable qualities. But here is where I want to play at the edges. What if we were 
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to state our most ineffable goals and attempt to get some indication of 
whether our students were achieving them? Let me be clear — no accreditor 
will ever jump on you for not stating the most ineffable goals. They’ll be 
happy with “critical thinking” or “literary analysis.” But maybe you won’t be 
satisfied. Maybe it would be good for your department to remind itself of 
these goals, work to articulate them, and get some indication about whether 
students are achieving them. So let’s explore what could happen. 

Articulating the Goals for Achieving the Ineffable 

First, you need a language for stating the ineffable. I want to cite here sev- 
eral sources for such a language. I’ll do so at some length — despite the danger 
that readers might be impatient — because this language feeds my teaching 
soul. I love to read it again, and I think other literature teachers do, too. 

Here is the language from the call for proposals for this book: 

[Students will] come to a new understanding of themselves, their 
world, and what might be a stake in the complex text before 
them. Such “sublime” experiences blur distinctions between sub- 
ject and object; they can involve self-loss, awe and even humility 
in the face of that which is other and/ or greater than the individ- 
ual reader. Ideally, the study of literature draws students out of 
their quotidian concerns and into perspectives to which they 
would otherwise not have access, introducing them to forms of 
experience they would not otherwise encounter. (Hciland and 
Rosenthal) 

Another wellspring from which we might draw language is the definition 
of “liberal learning” as shaped by the Association of American Colleges and 
Universities (AAC&U): 

A truly liberal education is one that prepares us to live responsible, 
productive, and creative lives in a dramatically changing world. It 
is an education that fosters a well-grounded intellectual resilience, 
a disposition toward lifelong learning, and an acceptance of 
responsibility for the ethical consequences of our ideas and 
actions. Liberal education requires that we understand the founda- 
tions of knowledge and inquiry about nature, culture and society; 
that we master core skills of perception, analysis, and expression; 
that we cultivate a respect for truth; that we recognize the impor- 
tance of historical and cultural context; and that we explore con- 
nections among formal learning, citizenship, and service to our 

communities Because liberal learning aims to free us from the 

constraints of ignorance, sectarianism, and myopia, it prizes 
curiosity and seeks to expand the boundaries of human knowl- 
edge. By its nature, therefore, liberal learning is global and plural- 
istic. It embraces the diversity of ideas and experiences that char- 
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acterize the social, natural, and intellectual world. To acknowledge 
such diversity in all its forms is both an intellectual commitment 
and a social responsibility, for nothing less will equip us to under- 
stand our world and to pursue fruitful lives. (“Statement”) 

Or use the language of a group of literary scholars under the aegis of the 
Modern Language Association which, supported by the Teagle Foundation, 
strove to articulate the relationship between literary study and the liberal arts: 

Delving into other languages and learning to read complex liter- 
ary texts rank among the most powerful means available for 
accomplishing [the] goals of liberal education and contributing to 

students’ personal and intellectual development 

Literary scholars explore how storytelling plays essential roles 
in all kinds of human comprehension. As students of literature 
learn about literary structure and form and the meanings of 
departures from established forms, they are acquiring the basic 
building blocks of understanding. At the same time, literature 
supplies an imaginative context through which readers gain 
insight into politics, history, society, emotion, and the interior life. 
Thus close reading of literary texts develops important analytic 
and interpretive skills that play central roles in complex human 
enterprises. What accomplished readers do with stories found in 
books — inhabit them, accept them provisionally as real, act 
according to their rules, tolerate their ambiguities, see their events 
from multiple and contradictory points of view, experience their 
bliss — informs what they can do with stories in the world at large. (4) 

Or turn to the language of “big questions” as employed by the Teagle 
Foundation and others such as the National Endowment for the Humanities’ 
“Enduring Questions” grant program. 


The Teagle Foundation has recently been probing into the big 
question of “Big Questions” in liberal education. We wanted to 
know whether a more direct engagement with the “Big Questions” 

would help invigorate students’ liberal education We haven’t 

tried to define those “Big Questions,” but we gave, as examples, 
such questions as “Who am I? What am I going to do with my 
life? What are my values? Is there such a thing as evil? What does 
it mean to be human? How can I understand suffering and 
death? What obligations do I have to others? What makes work, 
or a life, meaningful and satisfying?” We were also curious about 
shifting student attitudes (including their interest in religion and 
spirituality), about issues of value and meaning, and power and 
morality, and their place in undergraduate experience today — in 
the curriculum and beyond. (Connor) 
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How might we draw from this language to shape learning goals for litera- 
ture majors? Here are two possibilities, submitted as possible additions to the 
seven goals listed earlier: 

8. Students draw upon literature to contribute to their own search 
for meaning, their own engagement with the “big questions” of 
life and values — questions of life and death; good and evil; indi- 
vidual and society; power, transcendence, and virtue. 

9. Students come to a new understanding of themselves, their 
world, and what might be at stake in the complex text before 
them. They dare to explore new ideas and literary experiences. 

It’s bold to state goals 8 and 9. It’s not necessary for accreditation that you 
do so. You might not be able to get departmental agreement on any statement 
of such goals. But my point is that you can state these goals, and you can find 
ways to assess whether your students are reaching them. And doing so might 
be an incredibly rich and enlivening experience. 

If the Department Cannot Agree on Goals 

If the department cannot agree on a full set of goals, whether ordinary or 
ineffable, then generate a few goals you can agree on and move to Step Two — 
begin to collect information about how well students are meeting those goals. 
If disagreement or confusion about the goals is productive in helping the 
department clarify its mission and vision, then take the time to work through 
that discussion. But you should not let the department get bogged down in 
two years of bickering over the goals. In your report, you’ll say, “The depart- 
ment has agreed on this partial set of learning goals for its current assessment 
process. Further goals will be generated later.” The most important thing you 
can do, for yourselves and for the accreditors, is to show that you gathered 
some reasonable data and you acted on that information to make changes. 

Step Two: Select Measures of Student Learning 

Once you have a workable set of goals, you need to select what the assess- 
ment language calls “measures,” but what you can conceptualize as “indica- 
tors” of student learning. 

Direct and Indirect Measures 

One piece of jargon I think useful is the distinction between “direct” and 
“indirect” measures. A direct measure occurs when the student does some- 
thing — writes a paper, takes an exam, participates in class discussion — and 
someone directly observes and evaluates that performance. Indirect is every- 
thing else. Indirect measures involve some leap of inference between the stu- 
dent’s performance and the evaluation: for example, you ask students or alum- 
ni what they thought they learned, or you track their placement into jobs or 
graduate school. One assumes that students got the job or were admitted to 
graduate school because they had learned, but that is a leap of inference. 
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Indirect measures can be very useful, but you will want to have, and accredi- 
tors will urge, a mix of direct and indirect measures. 

The Basic, No-Frills System 

The most basic, no-frills system calls for: 

• One direct measure. The department examines at least a sample 
of students’ classroom work toward the end of their course of 
study. This can be done in two ways: 

— The faculty teaching courses that enroll significant numbers 
of seniors may report students’ strengths and weaknesses, 
based on the faculty members’ analysis of students’ class- 
room work. 

- And/ or a separate group of faculty may analyze a sample of 
senior student work to identify strengths and weaknesses. 

(A later section of this chapter gives more detail about how classroom work 
can be brought to the department for analysis and action.) 

• One indirect measure. The department gathers information from 
students via a student survey or focus groups. You can ask these 
three questions: 

— For each learning goal, how well did you achieve this goal 
(very well, somewhat well, not very well, not at all)? 

- What aspects of the department’s program, curriculum, 
courses, internships, or other activities in your major most 
helped you learn, and how did these things help (please be 
specific)? 

- What suggestions for improvement do you have in the 
department’s program? 

It is better to use two measures well than to proliferate measures you can- 
not use. Above all, do not list things in your reports that are not measures of 
learning for the program. For example, do not list the things you ask students 
to do (internship, senior research project) or assignments in a single course, 
unless the information about students’ strengths and weaknesses is brought to 
the department or to a committee for program-level discussion and action. 

Step Three: Use the Information for Improvement 

I suggest that you hold one two-hour meeting each year, in which the 
department, or a relevant committee, examines whatever information you 
have about student learning in one of your programs — say the undergraduate 
major. If the data are incomplete or inadequate, hold the meeting anyway, 
and devote part of the meeting to discussing how to get better data. 

By the end of the meeting, the department should identify one action 
item suggested by the data. For example, the data may suggest a number of 
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weaknesses in senior student work, but the department may choose to focus on 
one of them — helping students more effectively learn to employ more than 
one literary-critical lens or approach. Before the meeting ends, a person or 
small group is appointed to follow up. In the succeeding months, the depart- 
ment examines its curriculum to see where students are taught to use more 
than one literary-critical lens, where they are given practice and feedback, and 
where they develop the prerequisite skills they need. 

Taking Action 

Depending on what they find, departments will take action. The two most 
common actions are: 

• Curricular change: for example, emphasizing a particular skill 
more fully in one or more courses, adding/ dropping a course, 
changing prerequisites and requirements, or changing the 
sequence of courses. 

• Faculty development: for example, a series of brownbag lunches 
for faculty to share how they help students recognize multiple lit- 
erary-critical lenses, or how they encourage development of oral 
communication skills. 

Details: Using Classroom Work for Departmental Assessment 

For the direct measure, the department needs to look at a sample of stu- 
dent work, evaluate its strengths and weaknesses, and use that information for 
action at the department level. This is different from merely course-level grad- 
ing of student papers, from which the individual course instructor can make 
improvements in how she or he teaches. 

Begin with a sample of student work at the end of the course of study. 
Taking pre-post samples is more difficult than it sounds. It is better to start 
with an analysis of the strengths and weaknesses of student work toward the 
end of the program, select something to work on, and then perhaps go back 
for further information about students’ skills when they entered the program 
or when they completed the required American literature course. 

If you have a capstone for the major and/ or a thesis or exam for the mas- 
ter’s degree and PhD, these become ideal sites for gathering samples of stu- 
dent work. If you have no capstone, or if your community college English 
department contributes to an associate’s degree, you can take the work of stu- 
dents in two to three classes that enroll many or most of your students toward 
the end of their course of study. It is possible to take student work from a class 
that enrolls all levels, and extract only the work of seniors or students who 
have completed a specific number of credit hours, or students who have 
already taken a certain number of English courses. 

This is not an exercise in judging a single teacher. Rather, select student work 
that encompasses skills and knowledge the students have developed throughout 
their course of study. The quality of this work is everyone’s responsibility. The 
department works together as a team to address weaknesses and build on strengths. 
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Once you have a sample of student work, you need a way of analyzing it 
to identify strengths and weaknesses that the department can act on. There 
are two ways to get an analysis of students’ strengths and weaknesses. 

1. The instructor of the course conducts an analysis of students’ 
strengths and weaknesses and reports to the department. 

2. A group of department members analyzes a student work sample 
and reports to the department. 

Either of these methods needs a set of criteria against which the instruc- 
tor or the faculty readers can evaluate the student work. At minimum, the 
evaluators might work from a list of criteria related to the departmental learn- 
ing goals. A more detailed mode of analysis uses a “rubric” — that is, a partic- 
ular format for stating criteria and standards. In a rubric, the various traits of 
the work are evaluated separately, each with a scale from high-level work to 
low-level work. Fig. 1 is a rubric for literary-critical essays. 

You’ll see that it evaluates such qualities as complexity and originality. It 
deals intensively with critical thinking, as those skills appear in literary analy- 
sis. It would be sufficient for accreditation to use a rubric such as fig. 1 to eval- 
uate a sample of senior student literary-critical essays. However, in these 
essays, students may or may not achieve some of the more ineffable qualities 
stated in goals 8 and 9 above. 

So let’s ratchet up a notch, and talk about how to collect student work 
that might indicate whether students are achieving the ineffable goals 
expressed in 8 and 9. Let’s begin by looking at some student work that I 
believe exhibits some of the qualities expressed in Goals 8 and 9. It comes 
from my recent study of sixty-six highly effective teachers of introductory 
general education theology and religion classes at institutions both public and 
private ( Teaching and Learning ). Some of these faculty were experts at helping 
students address “big questions” without pushing students toward any partic- 
ular stance or blurring the boundaries between academic and religious 
instruction. Here is part of a student’s journal, from a public university gen- 
eral education course titled “Christianity and Cultures. ”2 The journal, by a 
student who chose to remain anonymous but gave permission for use of his 
work, was written in response to viewing a film about South Africa’s 
Reconciliation effort and its philosophy of “ubuntu.” Let’s ask whether the 
kinds of thinking reflected in this journal entry could be described as learn- 
ing goals, and then assessed. 

Ubuntu means: When you hurt others, you always hurt everyone, 
including yourself. Kant would call this a universal law, a law 
applicable at all times to all situations. The dcontological nature 
(having morality in one’s motives) of Ubuntu makes it interesting in 
that it is an internal quality and not a set of choices, and it seems 
that either people have it, or they don’t. Ubuntu is the real point of 
the movie, and they actually are testing you in the context of the 
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5 

4 

3 

2 

1 

Thesis 

The thesis of the paper is clear, 
complex, and challenging. It 
does not merely state the obvi- 
ous or exactly repeat others' 
viewpoints, but creatively and 
thoughtfully opens up our 
thinking about the work. 

The thesis is both 
clear and reason- 
ably complex. 

The thesis of the 
paper is clear. It 
takes a stand on a 
debatable issue, 
though the thesis 
may be unimagi- 
native, largely a 
recapitulation of 
readings and class 
discussion, and/or 
fairly obvious. 

Thesis is rel- 
evant to the 
assignment. 
It is dis- 
cernible, but 
the reader 
has to work 
to under- 
stand it. 

Thesis is irrel- 
evant to the 
assignment 
and/or not 
discernible. 

Complexity 

and 

Originality 

The essay is unusually 
thoughtful, deep, creative, and 
far-reaching in its analysis. 

The writer explores the subject 
from various points of view, 
acknowledges alternative inter- 
pretations or literary-critical 
approaches, and recognizes 
the complexity of issues in lit- 
erature and in life. Other works 
we have read and ideas we 
have discussed are integrated 
as relevant. The essay shows a 
curious and reflective mind at 
work. 

The essay is 
thoughtful and 
extensive in its 
analysis. It 
acknowledges 
alternative inter- 
pretations/ 
approaches and 
recognizes com- 
plexity in litera- 
ture and in life. 
Some other 
works are inte- 
grated as relevant 

The writer goes 
somewhat beyond 
merely para- 
phrasing some- 
one else’s point of 
view or repeating 
what was dis- 
cussed in class. 

And/or the essay 
does not integrate 
other relevant 
works we have 
read. 

Writer 
moves only 
marginally 
beyond 
merely para- 
phrasing 
someone 
else’s point 
of view or 
repeats what 
was dis- 
cussed in 
class. 

The paper is 
mere para- 
phrase or rep- 
etition. 

Organization 

and 

Coherence 

The reader feels that the writer 
is in control of the direction 
and organization of the essay. 
The essay follows a logical 
line of reasoning to support its 
thesis and to deal with count- 
er-evidence and alternative 
viewpoints. Sub-points are 
fashioned so as to open up the 
topic in the most 
effective way. 

As for “5” but 
sub-points may 
not be fashioned 
to open up the 
topic in the most 
effective way. 

The reader feels 
that the writer is 
in control of the 
direction and 
organization of the 
essay most of the 
time. The essay 
generally follows 
a logical line of 
reasoning to sup- 
port its thesis. 

The essay 
has some 
discernible 
main points. 

The essay has 
no discernible 
plan of organ- 
ization. 

Evidence and 
Support 

The writer’s claims and inter- 
pretations are richly supported 
with evidence from the works 
we have read, secondary 
sources, and sensible reason- 
ing. The writer assumes the 
reader has read the work and 
does not need the plot repeat- 
ed, but the writer refers richly 
and often to the events and 
words of the literature to sup- 
port his/her points. 

As for “5” but the 
writer may briefly 
drop into mere 
plot summary. 

The writer’s claims 
and interpretations 
about the works 
are generally 
backed with at 
least some evi- 
dence from the 
works. The writer 
may briefly drop 
into mere plot 
summary. 

The writer’s 
claims are 
sometimes 
backed with 
evidence 
and/or the 
paper drops 
often into 
mere plot 
summary. 

The paper is 
primarily plot 
summary. 

Style 

The language is clear, precise, 
and elegant. It achieves a 
scholarly tone without sound- 
ing pompous. It is the authentic 
voice of a curious mind at 
work, talking to other readers of 
the literary work. 

The language is 
clear and precise. 

The language is 
understandable 
throughout. 

The language 
is sometimes 
confusing. 
Sentences do 
not track. 

The language 
is often confus- 
ing. Sentences 
and paragraphs 
do not track. 


346 




HOW TO CONSTRUCT A SIMPLE, SENSIBLE, USEFUL DEPARTMENTAL ASSESSMENT PROCESS 



5 

4 

3 

2 

1 

Sources 

The essay integrates second- 
ary sources smoothly. It 
quotes when the exact words 
of another author are impor- 
tant, and otherwise paraphras- 
es. It does not just string 
together secondary sources, 
but uses them to support the 
writer’s own thinking. Each 
source is identified in the text, 
with some statement about its 
author; there are no quotes 
just stuck into the text without 
explanation. 

As for “5" but 
sources may 
occasionally be 
quoted with no 
contextual expla- 
nation. 

And/or writer 
may use direct 
quotation and 
paraphrase in 
less than optimal 
ways. 

The essay does 
not just string 
together second- 
ary sources, but 
uses them to sup- 
pod the writer’s 
own thinking. 

The essay 
strings 
together sec- 
ondary 
sources. 

There is no 
use of sec- 
ondary 
sources. 

Grammar and 
Punctuation 

There are no discernible 
departures from Edited 
Standard Written English 
(ESWE). 

There are a few 
departures from 
ESWE. 

There are no 
more than an 
average of 2 
departures from 
ESWE per page in 
the critical areas 
listed below. 

There are 
more than 2 
from ESWE 
per page in 
the critical 
areas listed 
below. 

Some portion 
of the essay is 
impossible to 
read because 
of departures 
from ESWE. 


Critical Areas: 

• Spelling or typographical errors 

• Sentence boundary punctuation (run-ons, comma splices, fused sentences, fragments) 

• Use of apostrophe, -s, and -es 

• Pronoun forms 

• Pronoun agreement, and providing antecedents for pronouns 

• Verb forms and subject-verb agreement 

• Use of gender-neutral language 

• Capitalization of proper nouns and of first words in the sentence 

Fig. 1. Rubric for Evaluating Student Literary-Critical Essays, Barbara E. Walvoord, date unknown. 


movie. They test you not on your critique of the confessors and 
aggressors or their victims or terrorists, but on your judgment of 
Anna Malon’s affair with Langston Whitfield. I fell prey to this as 
I should have, but soon realized that my very judgment and scoff- 
ing at the fact that I could NEVER forgive a woman that cheated 
on me, was in fact a measurement of my Ubuntu. Earlier in the 
movie I was trying to reconcile the philosophy of Ubuntu with my 
own beliefs, and especially my ego, and decided that it may fit to 
some degree, but I was wrong. When confronted with the idea 
(and past experience) of a woman cheating on me, I felt hate and 
anger, and wanted to hurt the woman . . . whom I am supposed to 
forgive. By understanding my level of Ubuntu I can better under- 
stand how they can forgive (but not forget) the atrocities commit- 
ted in South Africa; and I also recognize that my Ubuntu is scarce, 
at least at this point in my life. This is the true point of the movie, 
to make us take account of our own Ubuntu. 
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What are the ineffable qualities of this journal that we might want to 
encourage in other students? The student is 

• Relating the film to his own life, explaining the connections; 

• Bringing in material (Kant, deontology) from another course or 
other reading to help him think about big questions; 

• Reflecting an ongoing change in his thinking about the film, open 
to new ideas, new experiences, new directions; 

• Analyzing the themes and purposes of the film in relation to the 
“big questions” it poses. 


So how could we evaluate a sample of student journals? One simple 
option would be to take the four qualities above and simply look for their pres- 
ence in the journals. Readers could identify whether the quality appeared at 
all, whether it appeared frequently and habitually over a number of journal 
entries, and/ or whether it appeared in limited ways or more fully developed 
ways. Such an analysis would at least show the department what percentage of 
its senior students’ journals included these kinds of thinking. 

To be more precise, we could we construct a rubric for these journal 
entries. Fig. 2 is one attempt. 


Assignment: Journals are to record students' reflections about how the literature relates to their own search for 
meaning and values, and the way in which the literature may help them enter new perspectives, new worlds. 

Connecting Literature to “Big Questions” in Students’ Own Lives and Values; entertaining new perspectives and 
worldviews 

1 . Journal entries merely summarize/analyze the literature AND/OR merely reflect on the student’s struggle with 
“big questions" but make little or no explicit connection between the two. 

2. Journal entries summarize/analyze the literature AND reflect students’ struggle with “big questions." In at 
least one instance, the entry makes a connection between the two, but the connection is abbreviated, or it 
uses the literature in a simple way to draw “lessons" to apply to the student’s own life, or the student simply 
states that she agrees or disagrees. 

3. One entry makes thoughtful links between the literature and the student’s own struggle with big questions. It 
recognizes the complexity both of the literary work and of the “big questions." The journal entry reflects a 
willingness to entertain perspectives and understandings that are new to the student. 

4. More than one entry does as in 3 above. 

5. All of the entries do as in 3 above. The students’ musings are rich and deep, showing a thoughtful, reflective 
mind at work. 

Fig. 2. Rubric for Journals in English Literature, Barbara E. Walvoord, date unknown. 


A report to the department might show a table of rubric score averages, 
and/ or it might be a prose analysis of students’ strengths and weaknesses. 

This is my answer to the question we posed earlier about how to assess, 
and whether to assess, the ineffable values we often hold most dear. State those 
goals. Construct assignments that give you some indication of whether they 
are being achieved. State the criteria for assessing the assignment. Then exam- 
ine student work. Such a system is not perfect. Readers will not necessarily 
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Learning Goals (as listed earlier, numbers 1-8) 


Measure 

Goals 

Addressed 

How Information is Used 

Senior student literary-critical essays in 3 classes that enroll 
seniors are evaluated by the instructors of those 3 courses. 
Instructors may use a rubric or they may write an evaluation 
of students’ strengths and weaknesses in achieving the learn- 
ing goals intended by the assignment. 

12,3,4,5,6,7 

Instructors report to the department at 
the annual assessment meeting. 
Department reviews the evidence and 
takes action as needed. 

Annually, seniors enrolled in all classes are asked to com- 
plete a survey asking them how well they have achieved each 
of the learning goals, what aspects of the program helped 
them, and what suggestions they have for improvement. 

1-9 

Survey results are reported to the 
department at the annual assessment 
meeting, as above 

Faculty teaching the required American Literature course 
require student journals or online discussion boards that ask 
students to apply literature to their own search for meaning 
and values. A sample of these journals are read and dis- 
cussed each summer by the instructors of the course, who 
look for evidence of these kinds of learning, and who present 
an analysis to the department. 

8,9 

Reader’s report is presented to the 
department at the annual assessment 
meeting, as above. 


Conclusion of the assessment report for accreditors who need to know that the department is doing assessment: 

Examples of changes based on assessment data 

• Three years ago, at the annual meeting, faculty decided to focus on senior students' difficulties in goal number 3, using varied 
theoretical and literary-critical approaches. An examination of the students’ essays revealed several aspects to this problem. Two 
focus groups of students in two different senior-level courses provided further insights. The department examined its curriculum 
to identify classes where students were asked, within the same class, to apply two or more literary-critical approaches to a single 
piece of literature. The department then made changes in certain courses that would provide more instruction, practice, and feed- 
back in that learning goal. 

Alternate conclusion of the report for planning, budgeting and program review bodies that need to know what 

the department has most recently found and what are its plans and budgetary needs: 

Conclusions and plans 

The department, on the basis of its most recent assessment meeting, has decided to focus on students’ ability to achieve goals 8 

and 9. We plan to take the following action: 

• Hold four soup & brown-bag lunches. At the first meeting, Professor Haswold will present a review of the literature on this 
topic, particularly the work of the Teagle Foundation and the current programs sponsored by the AAC&U. At the next meeting, 
Professor Kim Graham, of Anyville University, will speak to our department faculty about how to support students’ search for 
meaning without violating church-state separation or our own academic and scholarly principles. At the third meeting, depart- 
ment members will share ideas with one another and examine strategies that seem to work in our own classrooms. At the final 
meeting, we will discuss possible implications for department-wide action. 

- Cost: 

• Soup and drinks for four lunches 

• Honorarium and mileage to Professor Graham 

Fig. 3. Sample Departmental Assessment Report, Barbara E. Walvoord, date unknown. 


evaluate a student journal in the same way. You’ll be aware that you are only 
viewing a whiff of smoke from the fire you hope burns within the student’s 
heart. But it is something. It makes the department go beyond the groundless 
spinning of words, to ask, “Do we have any indication that students are achiev- 
ing what we hope?” The ensuing faculty discussion, and the sharpened atten- 
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tion to these goals by individual faculty in their classrooms, may be the most 
valuable outcomes. 

Reporting Your Assessment System 

How does a department report its assessment system? Usually for two 
audiences: 

• Accreditors and others who need to know that the department is con- 
ducting assessment. 

• Administrators, budgeting and planning, and program reviewers 
who want to know what the department found and what it plans to do 
based on its assessment information. 

Fig. 3 shows a sample report with two possible endings — one for each audience. 

The sample report assumes the list of learning goals presented earlier, 
including the ineffable goals, numbers 8 and 9. It suggests some measures that 
might serve to indicate whether these goals were being achieved by students. It 
constructs some language by which the department might explain to outsiders 
what it does. 

Conclusion 

The point of this essay has been that assessment, while potentially danger- 
ous, can be helpful and sustainable if it is done sensibly. The wise department 
needs just three things for assessment: 

1 . A set of learning goals (at the end of this program, students will 
be able to...). 

2. Two measures that act as indicators of student learning: 

a. A direct measure: examine a sample of student work toward 
the end of their course of study. Identify strengths and weak- 
nesses. 

b. An indirect measure: ask students what they thought they 
learned, what helped them learn, and their suggestions for 
improvement. 

3. An annual meeting of two hours to consider the evidence about 
student learning and choose one item for action. 

Within this system, a department can find ways to articulate its most inef- 
fable goals for student learning and to gather indications about how well stu- 
dents are achieving them. 


RESOURCES ON ASSESSMENT 

Banta, Trudy W., cd. Assessing Student Learning in the Disciplines: Assessment Update 
Collections. San Francisco: Jossey, 2007. Print. 
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Suskie, Linda. Assesing Student Learning: A Common Sense Guide. 2nd cd. San 
Francisco: Jossey, 2009. Print. 

Walvoord, Barbara E. Assessment Clear and Simple: A Practical Guide for Institutions, 
Departments, and General Education. 2nd cd. San Francisco: Jossey, 2010. 
Print. 

Walvoord, Barbara E. and Virginia J. Anderson. Effective Grading: A Tool for 
Learning and Assessment in College. 2nd cd. San Francisco: Jossey, 2010. Print. 


NOTES 

1 For a definition of academic freedom, see Association of American 
Colleges and Universities, “Academic Freedom and Educational 
Responsibility.” 

~ The course is described in Teaching 122-28. 
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